Aug 2025· Conference on Empirical Methods in Natural Language Processing· pp. 9341-9370· 5 citations· 48 references
Computer Science
TL;DR
This work focuses on a practical yet challenging setting where only a small set of preference annotations can be collected per user, and proposes FaST, a highly parameter-efficient approach that leverages high-level features automatically discovered from the data, achieving the best overall performance.
Abstract
LLM-powered conversational assistants are often deployed in a one-size-fits-all manner, which fails to accommodate individual user preferences. Recently, LLM personalization -- tailoring models to align with specific user preferences -- has gained increasing attention as a way to bridge this gap. In this work, we specifically focus on a practical yet challenging setting where only a small set of preference annotations can be collected per user -- a problem we define as Personalized Preference Alignment with Limited Data (PPALLI). To support research in this area, we introduce two datasets -- DnD and ELIP -- and benchmark a variety of alignment techniques on them. We further propose FaST, a highly parameter-efficient approach that leverages high-level features automatically discovered from the data, achieving the best overall performance.
This work proposes RosePO, a framework to refine LLM-based recommendation through pairwise preference optimization with personalized smoothing, and incorporates a personalized smoothing factor predicted by a user oracle into the optimization objective.
Jia-Yi Liao, Xiang-Nan He, Ruo-Bing Xie et al.· ACM Transactions on Informat...· 0 citations
Aplaud (Adaptive Personalized Low-rank and User-specific Nested Decomposition), a lightweight and scalable framework for LLM personalization that outperforms state-of-the-art LoRA-based personalized LLM approaches in both generalization and inference efficiency.
Xin-Yu Li, Ruo-Ming Jin, Jian-Feng Zhu et al.· 1 citation
PALRec is proposed, a parameter-preserving augmentation framework that equips an LLM with recommendation capabilities while keeping its original parameters fixed and consistently outperforms fully fine-tuned counterparts in recommendation accuracy while preserving the LLM’s pre-trained knowledge.
Hyunsoo Na, Minseok Gang, Sang-goo Lee et al.· ACM Transactions on Informat...· 0 citations
CAIRO, a user context-aware item profiling framework for LLM-based reranking, first structures raw metadata and reviews into objective features and subjective traits, and employs a lightweight profiler to select the most relevant information for each user-item pair with limited serving-time overhead.
Dojun Hwang, Seunghan Lee, Cheonyoung Park et al.· 0 citations
This paper investigates how language models encode preference information in their intermediate representations, finding that activations from chosen and rejected responses form distinct clusters across layers, even in pretrained models.
Alessio Galatolo, Meriem Beloucif· 0 citations
Related blog posts
MIT News · Artificial Intelligence· news.mit.eduNov 19, 2026
An MIT and Dana-Farber Institute-led team used large language models to examine nearly 4,000 studies and hundreds of code repositories, revealing gaps in the information needed to scrutinize and reuse published research.
By studying the computational foundations of language learning, Associate Professor Jacob Andreas builds intelligent systems that can communicate effectively with humans.