1. Longitudinal Impact of Preference Biases on Recommender Systems' Performance.
- Author
-
Zhou, Meizi, Zhang, Jingjing, and Adomavicius, Gediminas
- Abstract
Recommender systems are ubiquitous on various online platforms and provide significant value to the users in helping them find relevant content/items to consume. After item consumption, users can often provide feedback (i.e., their preference ratings for the item) to the system. Research studies have shown that recommender systems' predictions, observed by users, can cause biases in users' postconsumption preference ratings. Because these ratings are typically fed back to the system as training data for future predictions, this process is likely to influence the system's performance over time. We use a simulation approach to investigate the longitudinal impact of preference biases on the dynamics of recommender systems' performance. Our results reveal that preference biases significantly impair recommendation performance and users' consumption outcomes, and larger biases cause disproportionately large negative effects. Additionally, less popular and less distinctive (in terms of their content) items are more susceptible to preference biases. Furthermore, considering the substantial impact of preference biases on recommendation performance, we examine the issue of debiasing user-submitted ratings. We find that relying solely on historical rating data is unlikely to be effective in debiasing; thus, we propose/evaluate new debiasing approaches that use additional relevant information that can be collected by recommendation platforms. Research studies have shown that recommender systems' predictions that are observed by users can cause biases in users' postconsumption preference ratings. This can happen as part of the standard, normal system use, where biases are typically caused by the system's inherent prediction errors (i.e., because of the less-than-perfect accuracy of recommendation methods). Because users' preference ratings are typically fed back to the system as training data for future predictions, this process is likely to influence the performance of the system in the long run. We use a simulation approach to study the longitudinal impact of preference biases (and their magnitude) on the dynamics of recommender systems' performance. Our simulation results show that preference biases significantly impair the system's prediction performance (i.e., prediction accuracy) as well as users' consumption outcomes (i.e., consumption relevance and diversity) over time. The impact is nonlinear to the size of the bias, that is, large bias causes disproportionately large negative effects. Also, items that are less popular and less distinctive (in terms of their content) are affected more by preference biases. Furthermore, given the impact of preference bias on the recommender systems' performance, we explore the problem of debiasing user-submitted ratings. We empirically demonstrate that relying solely on historical rating data is unlikely to be effective in debiasing. We also propose and evaluate two debiasing approaches that take into account additional relevant information that can be collected by recommendation platforms. Our findings provide important implications for the design of recommender systems. History: Olivia Liu Sheng, Senior Editor; Huimin Zhao, Associate Editor. Supplemental Material: The e-companion is available at https://doi.org/10.1287/isre.2021.0133. [ABSTRACT FROM AUTHOR]
- Published
- 2024
- Full Text
- View/download PDF