Do Personalized LLMs Need the Whole Conversation? Matching the Full History with Only 5% of Its Text via Relevance-Filtered Context

Abstract

Personalized assistants accumulate long conversation histories, and passing the whole history to an LLM with every request is costly. We show that it is also unnecessary. On PersonaMem-v2 (180 questions), with GPT-5.6-terra and Claude Opus 5.5 as judges, a relevance-filtered context, built by sentence-level retrieval that ranks the history by similarity to the current request and stops at a fixed budget, matches full-history accuracy with only 5% of the text, cutting the judge's input about 20-fold. An LLM summary of the same length performs on par, but it needs an LLM to read the entire history, whereas retrieval needs only a small embedding model. While both sentence-level retrieval and LLM summary perform strongly, retrieval does so while placing the relevant evidence in the context for only 61% of the questions, and an oracle given only that evidence beats the full history by 15.6 points, making better context selection a promising path to surpassing the full history.