Skip to content

FaST: Feature-aware Sampling and Tuning for Personalized Preference Alignment with Limited Data

Aug 2025 · Conference on Empirical Methods in Natural Language Processing · pp. 9341-9370 · 5 citations · 48 references
Computer Science

TL;DR

This work focuses on a practical yet challenging setting where only a small set of preference annotations can be collected per user, and proposes FaST, a highly parameter-efficient approach that leverages high-level features automatically discovered from the data, achieving the best overall performance.

Abstract

LLM-powered conversational assistants are often deployed in a one-size-fits-all manner, which fails to accommodate individual user preferences. Recently, LLM personalization -- tailoring models to align with specific user preferences -- has gained increasing attention as a way to bridge this gap. In this work, we specifically focus on a practical yet challenging setting where only a small set of preference annotations can be collected per user -- a problem we define as Personalized Preference Alignment with Limited Data (PPALLI). To support research in this area, we introduce two datasets -- DnD and ELIP -- and benchmark a variety of alignment techniques on them. We further propose FaST, a highly parameter-efficient approach that leverages high-level features automatically discovered from the data, achieving the best overall performance.

View source

Similar papers

Open access Aug 2026

RosePO: Customized Preference Alignment in LLM-Based Recommendation

This work proposes RosePO, a framework to refine LLM-based recommendation through pairwise preference optimization with personalized smoothing, and incorporates a personalized smoothing factor predicted by a user oracle into the optimization objective.

Jia-Yi Liao, Xiang-Nan He, Ruo-Bing Xie et al. · 0 citations
#artificial intelligence Review Sep 2026

Aplaud: Adaptive Personalized Low-Rank Decomposition for User-Specific LLM

Aplaud (Adaptive Personalized Low-rank and User-specific Nested Decomposition), a lightweight and scalable framework for LLM personalization that outperforms state-of-the-art LoRA-based personalized LLM approaches in both generalization and inference efficiency.

Xin-Yu Li, Ruo-Ming Jin, Jian-Feng Zhu et al. · 1 citation
#large language models Review Open access Sep 2026

PALRec: Large Language Model-Based Sequential Recommendation with Parameter-Preserving Augmentation

PALRec is proposed, a parameter-preserving augmentation framework that equips an LLM with recommendation capabilities while keeping its original parameters fixed and consistently outperforms fully fine-tuned counterparts in recommendation accuracy while preserving the LLM’s pre-trained knowledge.

Hyunsoo Na, Minseok Gang, Sang-goo Lee et al. · 0 citations
Review Aug 2026

Profiling What Matters: Context-Aware Item Profiles from Large-Scale Metadata for LLM Recommenders

CAIRO, a user context-aware item profiling framework for LLM-based reranking, first structures raw metadata and reviews into objective features and subjective traits, and employs a lightweight profiler to select the most relevant information for each user-item pair with limited serving-time overhead.

Dojun Hwang, Seunghan Lee, Cheonyoung Park et al. · 0 citations

Related blog posts

MIT News · Artificial Intelligence Nov 17, 2026

Learning the language of machine learning

By studying the computational foundations of language learning, Associate Professor Jacob Andreas builds intelligent systems that can communicate effectively with humans.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.