Lower bound estimation of recommendation error through user uncertainty modeling.

• We propose a MoEP model to estimate the magic barrier of recommender systems. • The uncertainty parameters are estimated instead of being specified by an expert. • The barriers were validated by comparison with the results of SOTA approaches. In machine learning, the Bayesian error is the lower bound of the prediction error induced by data distribution. In recommender systems, this is also known as the magic barrier (MGBR). MGBR estimation is an important issue because the recommended data frequently contain considerable uncertainties that are difficult to quantify. It is possible to determine the extent to which the recommendation algorithm can be optimized by obtaining the MGBR for a given dataset. MGBR estimation generally requires real user ratings that are not affected by external factors such as human emotions and living environment, which can be extremely difficult or even impossible to gather. Existing theoretical approaches based on simple models, such as Gaussian distributions, have limited estimation capabilities. In this paper, we propose a more sophisticated mixture of exponential power (MoEP) model, which enables adaptive parameter selection for intricate uncertainty. To fit the distribution of the real data, we constructed a flexible learning model that automatically adjusts super- or sub-Gaussian uncertainties using the MoEP components. To select parameters adaptively, we employed an expectation-maximization algorithm to infer the parameters of the components. To estimate the MGBR, we explored an approach for calculating the lower bound of the prediction error under the guidance of a probability model. Experiments on the four datasets validated the rationality of the proposed method. The results show that the MGBR estimated using the new model is marginally lower than the prediction error of state-of-the-art algorithms. [ABSTRACT FROM AUTHOR]