Search

The Internet era is an era of information explosion. By 2022, the global Internet users have reached more than 4 billion, and the social media users have exceeded 3 billion. People face a lot of news content every day, and it is almost impossible to get interesting information by browsing all the news content. Under this background, personalized news recommendation technology has been widely used, but it still needs to be further optimized and improved. In order to better push the news content of interest to different readers, users' satisfaction with major news websites should be further improved. This study proposes a new recommendation algorithm based on deep learning and reinforcement learning. Firstly, the RL algorithm is introduced based on deep learning. Deep learning is excellent in processing large-scale data and complex pattern recognition, but it often faces the challenge of low sample efficiency when it comes to complex decision-making and sequential tasks. While reinforcement learning (RL) emphasizes learning optimization strategies through continuous trial and error through interactive learning with the environment. Compared with deep learning, RL is more suitable for scenes that need long-term decision-making and trial-and-error learning. By feeding back the reward signal of the action, the system can better adapt to the unknown environment and complex tasks, which makes up for the relative shortcomings of deep learning in these aspects. A scenario is applied to an action to solve the sequential decision problem in the news dissemination process. In order to enable the news recommendation system to consider the dynamic changes in users' interest in news content, the Deep Deterministic Policy Gradient algorithm is applied to the news recommendation scenario. Opposing learning complements and combines Deep Q-network with the strategic network. On the basis of fully summarizing and thinking, this paper puts forward the mode of intelligent news dissemination and push. The push process of news communication information based on edge computing technology is proposed. Finally, based on Area Under Curve a Q-Leaning Area Under Curve for RL models is proposed. This indicator can measure the strengths and weaknesses of RL models efficiently and facilitates comparing models and evaluating offline experiments. The results show that the DDPG algorithm improves the click-through rate by 2.586% compared with the conventional recommendation algorithm. It shows that the algorithm designed in this paper has more obvious advantages in accurate recommendation by users. This paper effectively improves the efficiency of news dissemination by optimizing the push mode of intelligent news dissemination. In addition, the paper also deeply studies the innovative application of intelligent edge technology in news communication, which brings new ideas and practices to promote the development of news communication methods. Optimizing the push mode of intelligent news dissemination not only improves the user experience, but also provides strong support for the application of intelligent edge technology in this field, which has important practical application prospects. [ABSTRACT FROM AUTHOR]

Recommendation systems are crucial in navigating the vast digital market. However, user data's dynamic and non-stationary nature often hinders their efficacy. Traditional models struggle to adapt to the evolving preferences and behaviours inherent in user interaction data, posing a significant challenge for accurate prediction and personalisation. Addressing this, we propose a novel theoretical framework, the non-stationary transformer, designed to effectively capture and leverage the temporal dynamics within data. This approach enhances the traditional transformer architecture by introducing mechanisms accounting for non-stationary elements, offering a robust and adaptable solution for multi-tasking recommendation systems. Our experimental analysis, encompassing deep learning (DL) and reinforcement learning (RL) paradigms, demonstrates the framework's superiority over benchmark models. The empirical results confirm our proposed framework's efficacy, which provides significant performance enhancements, approximately 8% in LogLoss reduction and up to 2% increase in F1 score with other attention-related models. It also underscores its potential applicability across accumulative reward scenarios with pure reinforcement learning models. These findings advocate adopting non-stationary transformer models to tackle the complexities of today's recommendation tasks. [ABSTRACT FROM AUTHOR]

Session-aware recommender systems capture user-specific preferences that emerge within multiple user sessions by leveraging the sequential nature of user interactions. Existing session-aware recommendation methods face challenges in finding the right balance between exploration and exploitation leading to less diverse recommendations and also suffering from overestimation bias. This bias problem refers to the tendency for value estimates to be higher than their true values resulting in slower convergence, suboptimal, and less diverse recommendations. This paper proposes a Double Deep Q-network based session-aware recommender system, DDQN-SaRS, which takes care of the overestimation bias and generates diverse recommendations capturing the user's dynamic interests. The proposed system works in two phases. The first phase generates embedding for users, items, and sessions using a Graph Convolutional Network (GCN). The obtained embedding vectors are then given to Double Deep Q-Network (DDQN), a Double Deep Reinforcement Learning (DDRL) technique for suggesting items of interest to the user(s) in the second phase. DDQN decouples the task of action selection and action evaluation by utilizing two networks viz. main and target networks and resolves the overestimation bias problem while maintaining diversity in recommendations. The proposed system learns recommendation policies and corresponding rewards from a pure offline setting. It is validated on two real-world datasets: Diginetica from the CIKM Cup Challenge 2016 and Retailrocket from the Kaggle competition. Experimental results show that our proposed system, DDQN-SaRS outperformed various baseline algorithms viz. S-POP, Item-KNN, GRU4Rec, STAMP, HRNN, NSAR,IDSR, and GNN-GNF. [ABSTRACT FROM AUTHOR]

Recommender systems are a key technology for many applications, such as e-commerce, streaming media, and social media. Traditional recommender systems rely on collaborative filtering or content-based filtering to make recommendations. However, these approaches have limitations, such as the cold start and the data sparsity problem. This survey paper presents an in-depth analysis of the paradigm shift from conventional recommender systems to generative pre-trained-transformers-(GPT)-based chatbots. We highlight recent developments that leverage the power of GPT to create interactive and personalized conversational agents. By exploring natural language processing (NLP) and deep learning techniques, we investigate how GPT models can better understand user preferences and provide context-aware recommendations. The paper further evaluates the advantages and limitations of GPT-based recommender systems, comparing their performance with traditional methods. Additionally, we discuss potential future directions, including the role of reinforcement learning in refining the personalization aspect of these systems. [ABSTRACT FROM AUTHOR]

Conversational recommender systems (CRS) have garnered significant attention in academia and industry because of their ability to capture user preferences via system questions and user responses. Typically, in a CRS, reinforcement learning (RL) is utilized to determine the optimal timing for requesting attribute information or suggesting items. However, existing methods consider user-preferred attributes independently and ignore that attributes may be of different importance to the same user, in the attribute and item selection phases, which limits the accuracy and interpretability of CRS. Inspired by this, we propose deep conversational path reasoning (DeepCPR), which involves constructing a reasoning path on a graph with a series of user-favored attributes. It utilizes the attention mechanism to thoroughly examine the connections between these attributes and provide improved explanations for which attributes to inquire about or which items to recommend. In DeepCPR, two deep-learning-based modules are proposed to realize attribute and item selection. In the first module, the sequence of attributes confirmed by the user in conversation is encoded with a gated graph neural network to obtain the user's long-term preference using a self-attentionmechanism for the selection of candidate attributes. In the second module, a self-attention approach with more appropriate strategies is developed to dynamically select candidate items. In addition, to achieve fine-grained user preference modeling, a recurrent neural network is employed to aggregate the sequence of attributes that interact with the users. Numerous experimental evaluations conducted on four real CRS datasets show that the proposed method significantly outperforms existing advanced methods in terms of conversational recommendations. [ABSTRACT FROM AUTHOR]

Recommender systems play a vital role in discovering contents of interest to users in this information explosion era. However, traditional recommender systems only consider user immediate feedback and tend to recommend similar items according to users' historical interactions. Moreover, in real-world online applications, they lack sufficient user interaction data. In this paper, we propose a novel interactive recommender system by using deep reinforcement learning, which can take both user immediate rewards and future rewards into account. In order to tackle the effect of interaction data insufficiency on recommendation performance, we leverage the knowledge and relation information among items in external knowledge graphs to enrich the item embedding. We concatenate the user representation and user top-l latest historical interactions as the state and feed into the Bi-LSTM model to capture user dynamic preferences. Extensive experiments on two real-world data sets demonstrate the effectiveness and generality of our proposed recommender system. [ABSTRACT FROM AUTHOR]

Nowadays, interactive recommendation systems (IRS) play a significant role in our daily life. Recently, reinforcement learning has shown great potential in solving challenging tasks in IRS, since it can focus on long-term profit and can capture the dynamic preference of users. However, existing RL methods for IRS have two typical deficiencies. First, most state representation models use left-to-right recurrent neural networks to capture the user dynamics, which usually fail to handle the long and noisy sequential data in real life. Second, an IRS always needs to handle millions of items, leading to a large discrete action space in RL settings, which has not been fully addressed by the inefficient existing works. To bridge these deficiencies, in this paper, we propose attention-based tree recommendation (ATRec), an efficient tree-structured policy with attention-based state representation for IRS. ATRec uses an attention-based state representation model to effectively capture the user's dynamic preference hidden in the long and noisy sequence of behaviors. Moreover, to improve the learning efficiency, we propose an efficient tree-structured policy representation method, in which a complete tree is devised to represent the policy, and a novel parameter-sharing strategy is introduced. Extensive experiments are conducted on three real-world datasets and the results show the proposed ATRec obtains 42.3% improvement over some of the state of the arts methods in the hit rate and 21.4% improvement in the mean reciprocal rank of the top 30 ranked items. Additionally, the learning and decision efficiency can also be improved at an average of 35.5%. [ABSTRACT FROM AUTHOR]

Recommending personalized content from massive data for users is the key function of the recommendation system. The recommendation process of the traditional recommendation systems is often regarded as static, which cannot reflect the changes of user's real-time interest. This paper addressed this problem and presented a recommendation model that leverages the ability of deep learning methods to effectively deal with decision-making problems. In this model, a state generation module containing gate recurrent unit (GRU) and attention network was designed to obtain user's long and short-term preferences as well as history scores. Then, an actor-critic algorithm was employed to imitate the real-time recommendations. We trained the proposed model and evaluated it on four wellknown public datasets. It is proved that the proposed model is superior to existing recommendation models. [ABSTRACT FROM AUTHOR]

Recommender systems have become increasingly popular due to the significant rise in digital information over the internet in recent users. They help provide personalized recommendations to the user by selecting a few items out of a large set of items. However, with the growing size of item space and users, scalability remains a key issue for recommender systems. However, most existing policy gradient approaches in recommendations suffer from high variance leading to an increase in instability during the learning process. Policy Gradient Algorithms such as PPO are proven to be effective in large action spaces (a large number of items) as they learn the optimal policy directly from the samples. We use the PPO algorithm to train our Reinforcement Learning agent modeling the collaborative filtering process as a Markov Decision Process. PPO utilizes the actor-critic framework and thus mitigates the high variance in Policy Gradient Algorithms. Further, we address the cold start issue in Collaborative filtering with autoencoder-based content filtering. Proximal Policy Optimization (PPO) methods are today considered among the most effective reinforcement learning methods, achieving state-of-the-art performance and even outperforming Deep Q learning methods. In this paper, we propose a switching hybrid recommender system using the two different recommender system techniques. A switching hybrid system can switch between recommendation techniques depending on some criterion and can tackle its constituent recommender system's shortfall using the other counterpart in a particular situation. We show that our method outperforms various baseline methods on the popular Movielens datasets for different evaluation metrics. On Movielens 1m, our method outperforms the baseline by 9.19% in terms of R@10 and 3.86% and 6.58% in terms of P@10 and P@20, respectively. For the Movielens 100k dataset, our method improves on the baseline methods by 4.10% in terms of P@10 and 3.90% and 2.40% in terms of R@10 and R@20. [ABSTRACT FROM AUTHOR]

Although reinforcement learning (RL) can solve complex tasks after training, it's difficult to extend trained agents to target environments with perturbations. This lack of generalization ability has caused the dilemma of large-scale application of RL. Rich neural networks have brought huge improvements to deep learning, but these aren't adapted to RL and even bring negative effects. Therefore, RL algorithms have few choices of neural network, which greatly limits the representation and generalization ability of RL. To overcome these limitations, we propose a deep semi-dense compression network (DSCN) to improve RL generalization ability. First, we perform a structural extension on a general network model for RL. Then based on the information theory, we propose a semi-dense connection to enhance the neural information flow (NIF) of initial features. Finally, drawing on the ideas of course learning, we propose a channel compression approach to filter the redundant information of the deep network. In addition, we innovatively extend the experimental environment and evaluation metrics of the existing platform, which can fully evaluate the performance of DSCN. The experimental results show that our model achieves stable and significant generalization performance improvement. [ABSTRACT FROM AUTHOR]

Path-based relational reasoning over knowledge graphs has become increasingly popular due to a variety of downstream applications such as question answering in dialogue systems, fact prediction, and recommendation systems. In recent years, reinforcement learning (RL) based solutions for knowledge graphs have been demonstrated to be more interpretable and explainable than other deep learning models. However, the current solutions still struggle with performance issues due to incomplete state representations and large action spaces for the RL agent. We address these problems by developing HRRL (Heterogeneous Relational reasoning with Reinforcement Learning), a type-enhanced RL agent that utilizes the local heterogeneous neighborhood information for efficient path-based reasoning over knowledge graphs. HRRL improves the state representation using a graph neural network (GNN) for encoding the neighborhood information and utilizes entity type information for pruning the action space. Extensive experiments on real-world datasets show that HRRL outperforms state-of-the-art RL methods and discovers more novel paths during the training procedure, demonstrating the explorative power of our method. • Proposing a method to address the multi-hop reasoning problem. • Improving the choice of the action using the heterogeneous context. • Leveraging the neighborhood context for improving the reasoning precision. • Comprehensive evaluation of our model against state-of-the-art methods. [ABSTRACT FROM AUTHOR]

In precision medicine, the ultimate goal is to recommend the most effective treatment to an individual patient based on patient‐specific molecular and clinical profiles, possibly high‐dimensional. To advance cancer treatment, large‐scale screenings of cancer cell lines against chemical compounds have been performed to help better understand the relationship between genomic features and drug response; existing machine learning approaches use exclusively supervised learning, including penalized regression and recommender systems. However, it would be more efficient to apply reinforcement learning to sequentially learn as data accrue, including selecting the most promising therapy for a patient given individual molecular and clinical features and then collecting and learning from the corresponding data. In this article, we propose a novel personalized ranking system called Proximal Policy Optimization Ranking (PPORank), which ranks the drugs based on their predicted effects per cell line (or patient) in the framework of deep reinforcement learning (DRL). Modeled as a Markov decision process, the proposed method learns to recommend the most suitable drugs sequentially and continuously over time. As a proof‐of‐concept, we conduct experiments on two large‐scale cancer cell line data sets in addition to simulated data. The results demonstrate that the proposed DRL‐based PPORank outperforms the state‐of‐the‐art competitors based on supervised learning. Taken together, we conclude that novel methods in the framework of DRL have great potential for precision medicine and should be further studied. [ABSTRACT FROM AUTHOR]

In recent years, the recommendation system and robot learning are undoubtedly the two most popular application fields, and the core algorithms supporting these two fields are deep learning based on perception and reinforcement learning based on exploration learning, respectively. How to combine these two fields to better improve the development of the whole machine learning field is the dream of numerous researchers. The Deep Reinforcement Network (DRN) model successfully embedded reinforcement learning into the recommendation system, which provided a good idea for subsequent researchers. However, the disadvantage is also obvious, that is, the DRN model is built for news recommendations, meaning that the DRN model is not transferable, which is also the defect of many current recommendation system models. Meanwhile, the agent learning method adopted by the DRN model is primitive and inefficient. Among many models and algorithms that have emerged in recent years, we use the newly proposed deployment efficiency to measure their comprehensive quality and found that few models focus on both efficiency and performance improvement. To fill the gap of model deployment efficiency neglected by many researchers and to create a model of reinforcement learning agents with stronger performance, we have been exploring and trying to complete research on the Gate Attentional Factorization Machines (GAFM) model. Finally, we successfully integrated the GAFM model and reinforcement learning. The Deep Reinforcement Factorization Machines (DRFM) model proposed in this paper is based on the combination of deep learning with high perception ability and reinforcement learning with high exploration ability, centered on improving the deployment efficiency and learning performance of the model. The GAFM model is modified and upgraded using multidisciplinary techniques, and a new model-based random exploration strategy is proposed to update and optimize the recommendation list efficiently. Through parallel contrast experiments on various datasets, it is proved that the DRFM model surpasses the traditional recommendation system model in all aspects. The DRFM model is far superior to other models in terms of performance and robustness, and also significantly improved in terms of deployment efficiency. At the same time, we conduct a comparative analysis with the latest deep reinforcement learning algorithm and prove the unique advantages of the DRFM model. [ABSTRACT FROM AUTHOR]

In recent years, studies have revealed that introducing knowledge graphs (KGs) into recommendation systems as auxiliary information can improve recommendation accuracy. However, KGs are usually based on third-party data that may be manipulated by malicious individuals. In this study, we developed a poisoning attack strategy applied on a KG-based recommendation system to analyze the influence of fake links. The aim of an attacker is to recommend specific products to improve their visibility. Most related studies have focused on adversarial attacks on graph data; KG-based recommendation systems have rarely been discussed. We propose an attack model corresponding to recommendations. In the model, the current recommended status and a specified item are analyzed to estimate the effects of different attack decisions (addition or deletion of facts), thereby generating the optimal attack combination. Finally, the KG is contaminated by the attack combination so that the trained recommendation model recommends a specific item to as many people as possible. We formulated the process into a deep reinforcement learning method. Conducting experiments on the movie and the fund data sets enabled us to systematically analyze our poisoning attack strategy. The experimental results proved that the proposed strategy can effectively improve an item's ranking in a recommendation list. [ABSTRACT FROM AUTHOR]

Vehicle mobility optimization in urban areas is a long-standing problem in smart city and spatial data analysis. Given the complex urban scenario and unpredictable social events, our work focuses on developing a mobile sequential recommendation system to maximize the profitability of vehicle service providers (e.g., taxi drivers). In particular, we treat the dynamic route optimization problem as a long-term sequential decision-making task. A reinforcement-learning framework is proposed to tackle this problem, by integrating a self-check mechanism and a deep neural network for customer pick-up point monitoring. To account for unexpected situations (e.g., the COVID-19 outbreak), our method is designed to be capable of handling related environment changes with a self-adaptive parameter determination mechanism. Based on the yellow taxi data in New York City and vicinity before and after the COVID-19 outbreak, we have conducted comprehensive experiments to evaluate the effectiveness of our method. The results show consistently excellent performance, from hourly to weekly measures, to support the superiority of our method over the state-of-the-art methods (i.e., with more than 98% improvement in terms of the profitability for taxi drivers). [ABSTRACT FROM AUTHOR]

The world has seen major developments in the field of e-learning and distance learning, especially during the COVID-19 crisis, which revealed the importance of these two types of education and the fruitful benefits they have offered in a group of countries, especially those that have excellent infrastructure. At the Faculty of Sciences Semlalia, Cadi Ayyad University Marrakech, Morocco, we have created a simple electronic platform for remote practical work (RPW), and its results have been good in terms of student interaction and even facilitating the employment of a professor. The objective of this work is to propose a recommendation system based on deep quality-learning networks (DQNs) to recommend and direct students in advance of doing the RPW according to their skills of each mouse or keyboard click per student. We are focusing on this technology because it has strong, tremendous visibility and problem-solving ability that we will demonstrate in the result section. Our platform has enabled us to collect a range of students' and teachers' information and their interactions with the learning content we will rely on as inputs (a large number of images per second for each mouse or keyboard click per student) into our new system for output (doing the RPW). This technique is reflected in an attempt to embody the virtual teacher's image within the platform and then adequately trained with DQN technology to perform the RPW. [ABSTRACT FROM AUTHOR]

Recently, the application of deep reinforcement learning in the recommender system is flourishing and stands out by overcoming drawbacks of traditional methods and achieving high recommendation quality. The dynamics, long-term returns, and sparse data issues in the recommender system have been effectively solved. But the application of deep reinforcement learning brings problems of interpretability, overfitting, complex reward function design, and user cold start. This study proposed a tag-aware recommender system based on deep reinforcement learning without complex function design, taking advantage of tags to make up for the interpretability problems existing in the recommender system. Our experiment is carried out on the MovieLens dataset. The result shows that the DRL-based recommender system is superior than traditional algorithms in minimum error, and the application of tags have little effect on accuracy when making up for interpretability. In addition, the DRL-based recommender system has excellent performance on user cold start problems. [ABSTRACT FROM AUTHOR]

Reinforcement learning (RL) techniques have recently been introduced to recommender systems. Most existing research works focus on designing policy and learning algorithms of the recommender agent but seldom care about the top-aware issue , i.e., the performance on the top positions is not satisfying, which is crucial for real applications. To address the drawback, we propose a Supervised deep Reinforcement learning Recommendation framework named as SRR. Within this framework, we utilize a supervised learning (SL) model to partially guide the learning of recommendation policy, where the supervision signal and RL signal are jointly employed and updated in a complementary fashion. We empirically find that suitable weak supervision helps to balance the immediate reward and the long-term reward, which nicely addresses the top-aware issue in RL based recommendation. Moreover, we perform a further investigation on how different supervision signals impact on recommendation policy. Extensive experiments are carried out on two real-world datasets under both the offline and simulated online evaluation settings, and the results demonstrate that the proposed methods indeed resolve the top-aware issue without much performance sacrifice in the long-run, compared with the state-of-the-art methods. [ABSTRACT FROM AUTHOR]

The design of recommendation strategies in the adaptive learning systems focuses on utilizing currently available information to provide learners with individual‐specific learning instructions. As a critical motivate for human behaviours, curiosity is essentially the drive to explore knowledge and seek information. In a psychologically inspired view, we propose a curiosity‐driven recommendation policy within the reinforcement learning framework, allowing for an efficient and enjoyable personalized learning path. Specifically, a curiosity reward from a well‐designed predictive model is generated to model one's familiarity with the knowledge space. Given such curiosity rewards, we apply the actor–critic method to approximate the policy directly through neural networks. Numerical analyses with a large continuous knowledge state space and concrete learning scenarios are provided to further demonstrate the efficiency of the proposed method. [ABSTRACT FROM AUTHOR]

Knowledge Graphs (KGs) have been applied to various application scenarios including Web searching, Q&A, recommendation system, natural language processing and so on. However, the vast majority of Knowledge Bases (KBs) are incomplete, necessitating a demand for KB completion (KBC). Methods of KBC used in the mainstream current knowledge base include the latent factor model, the random walk model and recent popular methods based on reinforcement learning, which performs well in their respective areas of expertise. Recurrent neural network (RNN) and its variants model temporal data by remembering information for long periods, however, whether they also have the ability to use the information they have already remembered to achieve complex reasoning in the knowledge graph. In this paper, we produce a novel framework (ALSTM) based on the Attention mechanism and Long Short-Term Memory (LSTM), which associates structure learning with parameter learning of first-order logical rules in an end-to-end differentiable neural networks model. In this framework, we designed a memory system and employed a multi-head dot product attention (MHDPA) to interact and update the memories embedded in the memory system for reasoning purposes. This is also consistent with the process of human cognition and reasoning, looking for enlightenment for the future in historical memory. In addition, we explored the use of inductive bias in deep learning to facilitate learning of entities, relations, and rules. Experiments establish the efficiency and effectiveness of our model and show that our method achieves better performance in tasks which include fact prediction and link prediction than baseline models on several benchmark datasets such as WN18RR, FB15K-237 and NELL-995. [ABSTRACT FROM AUTHOR]

In this article, we study a multi-step interactive recommendation problem for explicit-feedback recommender systems. Different from the existing works, we propose a novel user-specific deep reinforcement learning approach to the problem. Specifically, we first formulate the problem of interactive recommendation for each target user as a Markov decision process (MDP). We then derive a multi-MDP reinforcement learning task for all involved users. To model the possible relationships (including similarities and differences) between different users' MDPs, we construct user-specific latent states by using matrix factorization. After that, we propose a user-specific deepQ-learning (UDQN) method to estimate optimal policies based on the constructed user-specific latent states. Furthermore, we propose Biased UDQN (BUDQN) to explicitly model user-specific information by employing an additional bias parameter when estimating the Q-values for different users. Finally, we validate the effectiveness of our approach by comprehensive experimental results and analysis. [ABSTRACT FROM AUTHOR]

The workshop program of the Association for the Advancement of Artificial Intelligence’s 33rd Conference on Artificial Intelligence (AAAI-19) was held in Honolulu, Hawaii, on Sunday and Monday, January 27–28, 2019. There were fifteen workshops in the program: Affective Content Analysis: Modeling Affect-in-Action, Agile Robotics for Industrial Automation Competition, Artificial Intelligence for Cyber Security, Artificial Intelligence Safety, Dialog System Technology Challenge, Engineering Dependable and Secure Machine Learning Systems, Games and Simulations for Artificial Intelligence, Health Intelligence, Knowledge Extraction from Games, Network Interpretability for Deep Learning, Plan, Activity, and Intent Recognition, Reasoning and Learning for Human-Machine Dialogues, Reasoning for Complex Question Answering, Recommender Systems Meet Natural Language Processing, Reinforcement Learning in Games, and Reproducible AI. This report contains brief summaries of the all the workshops that were held [ABSTRACT FROM AUTHOR]

In this paper, we propose a framework based on Hierarchical Reinforcement Learning for dialogue management in a Conversational Recommender System scenario. The framework splits the dialogue into more manageable tasks whose achievement corresponds to goals of the dialogue with the user. The framework consists of a meta-controller, which receives the user utterance and understands which goal should pursue, and a controller, which exploits a goal-specific representation to generate an answer composed by a sequence of tokens. The modules are trained using a two-stage strategy based on a preliminary Supervised Learning stage and a successive Reinforcement Learning stage. [ABSTRACT FROM AUTHOR]

In light of the emergence of deep reinforcement learning (DRL) in recommender systems research and several fruitful results in recent years, this survey aims to provide a timely and comprehensive overview of recent trends of deep reinforcement learning in recommender systems. We start by motivating the application of DRL in recommender systems, followed by a taxonomy of current DRL-based recommender systems and a summary of existing methods. We discuss emerging topics, open issues, and provide our perspective on advancing the domain. The survey serves as introductory material for readers from academia and industry to the topic and identifies notable opportunities for further research. [ABSTRACT FROM AUTHOR]

Recommender systems (RSs) have been employed for many real-world applications including search engines, social networks, and information retrieval systems as powerful intelligent techniques for creating personalized recommendations. RSs mainly utilize different data resources such as user-item ratings, tag data, and social relationships in their recommendation process to overcome the data sparsity issue. One of the main challenges is how to integrate the results obtained by these data resources. Moreover, most of the existing RSs focus on the recommendation accuracy, and do not consider the reliability of the recommendations. To address these challenges, in this study, a reliable recommendation method is developed, which employs deep neural networks and reinforcement learning. The proposed method firstly uses a stacked denoising autoencoder model to obtain the latent features from the rating, tag, and trust data. The similarity value between each pair of users is then calculated in three different aspects according to the extracted latent features. An ensemble approach is developed by integrating the three similarities using reinforcement learning to determine the final similarity value. The ensemble similarity values are used in the rating prediction process, and the reliability of the predicted ratings is evaluated using an effective measure. Finally, a recommendation strategy is developed based on the integration of the predicted ratings and their reliability values. We evaluated the effectiveness of the proposed method by conducting numerous experiments on three datasets where the results show that the proposed method is far superior to other recommendation models in making more accurate recommendations. [ABSTRACT FROM AUTHOR]

Precise prescription of medication dosing is crucial to patients, especially among Intensive Care Unit (ICU) patients. However, improper administration of some sensitive therapeutic medications (e.g., heparin) might place patients at unneeded risk, even life-threatening. Numerous factors such as a patient's clinical phenotype, genotype, and environmental factors will affect the heparin dose response. As a result, it is challenging to prescribe the optimal initial dose of heparin. In this paper, an individualized dosing policy is proposed to determine the optimal initial dose and minimize the risk of mis-dosing, as well as preventing the patients from late complications associated with medications dosing. A latent batch-constrained deep reinforcement learning (RL) algorithm is proposed to guarantee the safety of the medication recommendation system. The agent can observe a latent representation of patents and generate medication dosing solutions in successive and limited action spaces. The individualized dosing policy aims to reduce the extrapolation errors in the off-policy algorithms, by evaluating over-dosing and under-dosing of heparin in patients. Our results evaluated on Medical Information Mart for Intensive Care III (MIMIC-III) database demonstrate that the latent batch-constrained RL algorithm can work effectively from the retrospective data, showing promise to be used in future medication dosing policies. [ABSTRACT FROM AUTHOR]

Knowledge graphs (KGs) can provide rich, structured information for recommendation systems as well as increase accuracy and perform explicit reasoning. Deep reinforcement learning (RL) has also sparked great interest in personalized recommendations. The combination of the two holds promise in carrying out interpretable causal inference procedures and improving the performance of graph-structured recommendation. However, most KG-based recommendation focus on rich semantic relationships between entities in a heterogeneous knowledge graph, and thus fail to fully make use of the image information corresponding to an entity. In order to address these issues, we proposed a novel Multi-modal Knowledge-aware Reinforcement Learning Network (MKRLN), which couples recommendation and interpretability by providing actual paths in multi-modal KG (MKG). The MKRLN can generate path representation by composing the structural and visual information of entities, and infers the underlying rational of agent-MKG interactions by leveraging the sequential dependencies within a path from the MKG. In addition, as KGs have too many attributes and entities, their combination with RL leads to too many action spaces and states in the reinforcement learning space, which complicates the search of action spaces. Furthermore, in order to solve this problem, we proposed a new hierarchical attention-path, which makes users focus their attention on the items they are interested in. This reduces the relations and entities in the KGs, which in turn reduces the action space and state in RL, shortens the path to the target entity, and improves the accuracy of recommendation. Our model has explicit explanation ability in knowledge and images. Finally, we extensively evaluated our model on several large-scale real-world benchmark datasets, and it yielded favorable results compared with state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Recommender systems aim to maximize the overall accuracy for long-term recommendations. However, most of the existing recommendation models adopt a static view, and ignore the fact that recommendation is a dynamic sequential decision-making process. As a result, they fail to adapt to new situations and suffer from the cold-start problem. Although sequential recommendation methods have been gaining attention recently, the objective of long-term recommendation still has not been explicitly addressed since these methods are developed for short-term prediction situations. To overcome these problems, we propose a novel top-N interactive recommender system based on deep reinforcement learning. In our model, the processes of recommendation are viewed as Markov decision processes (MDP), wherein the interactions between agent (recommender system) and environment (user) are simulated by the recurrent neural network (RNN). In addition, reinforcement learning is employed to optimize the proposed model for the purpose of maximizing long-term recommendation accuracy. Experimental results based on several benchmarks show that our model significantly outperforms previous top-N methods in terms of Hit-Rate and NDCG for the long-term recommendation, and can be applied to both cold-start and warm-start scenarios. • A novel top-N interactive recommender system based on deep reinforcement learning is proposed. • The interactions between recommender system and users are simulated by recurrent neural networks. • The proposed model can deal with both cold-start and warm-start scenarios. • Reinforcement learning and supervised learning are employed to optimize the proposed model for long-term recommendation accuracy. • Experiments and comparisons are conducted to show the merits of the proposed model. [ABSTRACT FROM AUTHOR]

Reinforcement learning techniques have recently been introduced to interactive recommender systems to capture the dynamic patterns of user behavior during the interaction with recommender systems and perform planning to optimize long-term performance. Most existing research work focuses on designing policy and learning algorithms of the recommender agent but seldom cares about the state representation of the environment, which is indeed essential for the recommendation decision making. In this paper, we first formulate the interactive recommender system problem with a deep reinforcement learning recommendation framework. Within this framework, we then carefully design four state representation schemes for learning the recommendation policy. Inspired by recent advances in feature interaction modeling in user response prediction, we discover that explicitly modeling user–item interactions in state representation can largely help the recommendation policy perform effective reinforcement learning. Extensive experiments on four real-world datasets are conducted under both the offline and simulated online evaluation settings. The experimental results demonstrate the proposed state representation schemes lead to better performance over the state-of-the-art methods. [ABSTRACT FROM AUTHOR]