Search

With the rise of short video platforms, video recommendation technology faces more complex challenges. Currently, there are multiple non-personalized modules in the video recommendation pipeline that urgently need personalized modeling techniques for improvement. Inspired by the success of uplift modeling in online marketing, we attempt to implement uplift modeling in the video recommendation scenario. However, we face two main challenges: 1) Design and utilization of treatments, and 2) Capture of user real-time interest. To address them, we design adjusting the distribution of videos with varying durations as the treatment and propose Coarse-to-fine Dynamic Uplift Modeling (CDUM) for real-time video recommendation. CDUM consists of two modules, CPM and FIC. The former module fully utilizes the offline features of users to model their long-term preferences, while the latter module leverages online real-time contextual features and request-level candidates to model users' real-time interests. These two modules work together to dynamically identify and targeting specific user groups and applying treatments effectively. Further, we conduct comprehensive experiments on the offline public and industrial datasets and online A/B test, demonstrating the superiority and effectiveness of our proposed CDUM. Our proposed CDUM is eventually fully deployed on the Kuaishou platform, serving hundreds of millions of users every day. The source code will be provided after the paper is accepted., Comment: 9 pages, 4 figures, 5 tables

In recommender systems, reinforcement learning solutions have shown promising results in optimizing the interaction sequence between users and the system over the long-term performance. For practical reasons, the policy's actions are typically designed as recommending a list of items to handle users' frequent and continuous browsing requests more efficiently. In this list-wise recommendation scenario, the user state is updated upon every request in the corresponding MDP formulation. However, this request-level formulation is essentially inconsistent with the user's item-level behavior. In this study, we demonstrate that an item-level optimization approach can better utilize item characteristics and optimize the policy's performance even under the request-level MDP. We support this claim by comparing the performance of standard request-level methods with the proposed item-level actor-critic framework in both simulation and online experiments. Furthermore, we show that a reward-based future decomposition strategy can better express the item-wise future impact and improve the recommendation accuracy in the long term. To achieve a more thorough understanding of the decomposition strategy, we propose a model-based re-weighting framework with adversarial learning that further boost the performance and investigate its correlation with the reward-based strategy., Comment: 12 pages, 8 figures

The bulk states of Hermitian systems are believed insensitive to local Hermitian impurities or perturbations except for a few impurity-induced bound states. Thus, it is important to ask whether \textit{local} non-Hermiticity can cause drastic changes to the original Hermitian systems. Here we address this issue affirmatively and present exact solutions for the double chain model with local non-Hermitian terms possessing parity-time ($\mathcal{PT}$) symmetry. Induced by the non-Hermiticity, the system undergoes a sequence of $\mathcal{PT}$-symmetry breakings, after which the eigenenergies appear in complex conjugate pairs. The associated extended bulk states then become scale-free localized and unidirectionally accumulated around the impurity. There exist mobility edges separating the residual extended states until a full scale-free localization of all eigenstates. Further increasing the non-Hermitity counter-intuitively brings the system to a $\mathcal{PT}$-restoration regime with fully real spectra except for a pair of complex bound states. We demonstrate that the local non-Hermiticity generated scale-free localization is a general phenomenon and can even survive the quasiperiodic disorder. Our results indicate that the bulk properties of the original Hermitian system can be globally reshaped by local non-Hermiticity., Comment: 14 pages, 11 figures, published version in Phys. Rev. B as Editors' Suggestion

The wide popularity of short videos on social media poses new opportunities and challenges to optimize recommender systems on the video-sharing platforms. Users sequentially interact with the system and provide complex and multi-faceted responses, including watch time and various types of interactions with multiple videos. One the one hand, the platforms aims at optimizing the users' cumulative watch time (main goal) in long term, which can be effectively optimized by Reinforcement Learning. On the other hand, the platforms also needs to satisfy the constraint of accommodating the responses of multiple user interactions (auxiliary goals) such like, follow, share etc. In this paper, we formulate the problem of short video recommendation as a Constrained Markov Decision Process (CMDP). We find that traditional constrained reinforcement learning algorithms can not work well in this setting. We propose a novel two-stage constrained actor-critic method: At stage one, we learn individual policies to optimize each auxiliary signal. At stage two, we learn a policy to (i) optimize the main signal and (ii) stay close to policies learned at the first stage, which effectively guarantees the performance of this main policy on the auxiliaries. Through extensive offline evaluations, we demonstrate effectiveness of our method over alternatives in both optimizing the main goal as well as balancing the others. We further show the advantage of our method in live experiments of short video recommendations, where it significantly outperforms other baselines in terms of both watch time and interactions. Our approach has been fully launched in the production system to optimize user experiences on the platform., Comment: Code Available at https://github.com/AIDefender/TSCAC. arXiv admin note: substantial text overlap with arXiv:2205.13248

Recently, short video platforms have achieved rapid user growth by recommending interesting content to users. The objective of the recommendation is to optimize user retention, thereby driving the growth of DAU (Daily Active Users). Retention is a long-term feedback after multiple interactions of users and the system, and it is hard to decompose retention reward to each item or a list of items. Thus traditional point-wise and list-wise models are not able to optimize retention. In this paper, we choose reinforcement learning methods to optimize the retention as they are designed to maximize the long-term performance. We formulate the problem as an infinite-horizon request-based Markov Decision Process, and our objective is to minimize the accumulated time interval of multiple sessions, which is equal to improving the app open frequency and user retention. However, current reinforcement learning algorithms can not be directly applied in this setting due to uncertainty, bias, and long delay time incurred by the properties of user retention. We propose a novel method, dubbed RLUR, to address the aforementioned challenges. Both offline and live experiments show that RLUR can significantly improve user retention. RLUR has been fully launched in Kuaishou app for a long time, and achieves consistent performance improvement on user retention and DAU.

We study the boundary-dissipated transverse field Ising model described by a Lindblad Master equation and exactly solve its Liouvillian spectrum in the whole parameter space. By mapping the Liouvillian into a Su-Schrieffer-Heeger model with imaginary boundary potentials under a parity constraint, we solve the rapidity spectrum analytically and thus construct the Liouvillian spectrum strictly with a parity constraint condition. Our results demonstrate that the Liouvillian spectrum displays four different structures, which are characterized by different numbers of segments. By analyzing the properties of rapidity spectrum, we can determine the phase boundaries between different spectrum structures analytically and prove the Liouvillian gap fulfilling a duality relation in the weak and strong dissipation region. Furthermore, we unveil the existence of a dynamical duality, i.e., the long-time relaxation dynamics exhibits almost the same dynamical behavior in the weak and strong dissipation region as long as the duality relation holds true., Comment: 15 pages, 10 figures

Few-shot classification aims to learn a model that can generalize well to new tasks when only a few labeled samples are available. To make use of unlabeled data that are more abundantly available in real applications, Ren et al. \shortcite{ren2018meta} propose a semi-supervised few-shot classification method that assigns an appropriate label to each unlabeled sample by a manually defined metric. However, the manually defined metric fails to capture the intrinsic property in data. In this paper, we propose a \textbf{S}elf-\textbf{A}daptive \textbf{L}abel \textbf{A}ugmentation approach, called \textbf{SALA}, for semi-supervised few-shot classification. A major novelty of SALA is the task-adaptive metric, which can learn the metric adaptively for different tasks in an end-to-end fashion. Another appealing feature of SALA is a progressive neighbor selection strategy, which selects unlabeled data with high confidence progressively through the training phase. Experiments demonstrate that SALA outperforms several state-of-the-art methods for semi-supervised few-shot classification on benchmark datasets., Comment: 9 pages

Knowledge graph (KG) embedding aims at learning the latent representations for entities and relations of a KG in continuous vector spaces. An empirical observation is that the head (tail) entities connected by the same relation often share similar semantic attributes -- specifically, they often belong to the same category -- no matter how far away they are from each other in the KG; that is, they share global semantic similarities. However, many existing methods derive KG embeddings based on the local information, which fail to effectively capture such global semantic similarities among entities. To address this challenge, we propose a novel approach, which introduces a set of virtual nodes called \textit{\textbf{relational prototype entities}} to represent the prototypes of the head and tail entities connected by the same relations. By enforcing the entities' embeddings close to their associated prototypes' embeddings, our approach can effectively encourage the global semantic similarities of entities -- that can be far away in the KG -- connected by the same relation. Experiments on the entity alignment and KG completion tasks demonstrate that our approach significantly outperforms recent state-of-the-arts., Comment: 11 pages, 6 figures, Knowledge Graph Embedding

Watch-time prediction remains to be a key factor in reinforcing user engagement via video recommendations. It has become increasingly important given the ever-growing popularity of online videos. However, prediction of watch time not only depends on the match between the user and the video but is often mislead by the duration of the video itself. With the goal of improving watch time, recommendation is always biased towards videos with long duration. Models trained on this imbalanced data face the risk of bias amplification, which misguides platforms to over-recommend videos with long duration but overlook the underlying user interests. This paper presents the first work to study duration bias in watch-time prediction for video recommendation. We employ a causal graph illuminating that duration is a confounding factor that concurrently affects video exposure and watch-time prediction -- the first effect on video causes the bias issue and should be eliminated, while the second effect on watch time originates from video intrinsic characteristics and should be preserved. To remove the undesired bias but leverage the natural effect, we propose a Duration Deconfounded Quantile-based (D2Q) watch-time prediction framework, which allows for scalability to perform on industry production systems. Through extensive offline evaluation and live experiments, we showcase the effectiveness of this duration-deconfounding framework by significantly outperforming the state-of-the-art baselines. We have fully launched our approach on Kuaishou App, which has substantially improved real-time video consumption due to more accurate watch-time predictions., Comment: 10 pages

We carry out a systematical study of the size scaling of Liouvillian gap in boundary-dissipated one-dimensional quasiperiodic and disorder systems. By treating the boundary-dissipation operators as a perturbation, we derive an analytical expression of the Liouvillian gap, which indicates clearly the Liouvillian gap being proportional to the minimum of boundary densities of eigenstates of the underlying Hamiltonian, and thus give a theoretical explanation why the Liouvillian gap has different size scaling relation in the extended and localized phase. While the Liouvillian gap displays a power-law size scaling $\Delta_{g}\propto L^{- 3}$ in the extended phase, our analytical result unveils that the Liouvillian gap fulfills an exponential scaling relation $\Delta_{g}\propto e^{- \kappa L}$ in the localized phase, where $\kappa$ takes the largest Lyapunov exponent of localized eigenstates of the underlying Hamiltonian. By scrutinizing the extended Aubry-Andr\'{e}-Harper model, we numerically confirm that the Liouvillian gap fulfills the exponential scaling relation and the fitting exponent $\kappa$ coincides pretty well with the analytical result of Lyapunov exponent. The exponential scaling relation is further verified numerically in other one-dimensional quasiperiodic and random disorder models. We also study the relaxation dynamics and show the inverse of Liouvillian gap giving a reasonable timescale of asymptotic convergence to the steady state., Comment: 11 pages, 4 figures