1. User Moderling, Personalization, and Personalized Question Generation in Open-Domain Dialogue Systems
- Author
-
Bowden, Kevin, Walker, Marilyn1, Bowden, Kevin, Bowden, Kevin, Walker, Marilyn1, and Bowden, Kevin
- Abstract
Research on open-domain social dialogue systems has exploded over the last few years with the advent of large language models (LLMs) that can chat about any topic. Unlike traditional dialogue systems, open-domain dialogue systems cannot assume any specific information need or domain restrictions - the only inherent goal is to converse socially. While modern systems have access to more information and better tools, foundational components of natural human-human conversation remain elusive, i.e., intimacy and agency. In this thesis, we hypothesize that personalization is pivotal in fostering this genuine connection between users and open-domain dialogue systems.Our first hypothesis is that personalizing the conversation to specific user interests will build a sense of understanding, rapport, and agency. To investigate this, we heuristically combine the results of an extensive natural language understanding pipeline with handcrafted rules to build a user modeling mechanism; this user model then personalizes the experience through response adaptation and topic-promotion strategies, resulting in a statistically significant positive impact on perceived conversation quality and length when evaluated at scale with a testbed open-domain dialogue system, that real Amazon Echo users access. Analyzing the user models unveils nuanced insights into user preferences, emphasizing a desire for more personalized experiences and receptiveness toward personal questions. This leads to our second hypothesis - asking appropriate personalized follow-up questions (PQs) helps to create a more engaged user experience that increases user satisfaction. Our initial test of this hypothesis uses a crowdsourced corpus of PQs (Would You Rather and Hypothetical) in the testbed system's dialogue policy. Our evaluation of the policy shows that it results in extended topical depth, leading to statistically significant longer, more highly rated conversations.However, crowdsourcing PQs for every user inte
- Published
- 2024