Back to Search Start Over

Online Personalizing White-box LLMs Generation with Neural Bandits

Authors :
Chen, Zekai
Daniel, Weeden
Chen, Po-yu
Buet-Golfouse, Francois
Publication Year :
2024

Abstract

The advent of personalized content generation by LLMs presents a novel challenge: how to efficiently adapt text to meet individual preferences without the unsustainable demand of creating a unique model for each user. This study introduces an innovative online method that employs neural bandit algorithms to dynamically optimize soft instruction embeddings based on user feedback, enhancing the personalization of open-ended text generation by white-box LLMs. Through rigorous experimentation on various tasks, we demonstrate significant performance improvements over baseline strategies. NeuralTS, in particular, leads to substantial enhancements in personalized news headline generation, achieving up to a 62.9% improvement in terms of best ROUGE scores and up to 2.76% increase in LLM-agent evaluation against the baseline.<br />Comment: 7 pages

Details

Database :
arXiv
Publication Type :
Report
Accession number :
edsarx.2404.16115
Document Type :
Working Paper