Long-term memory enables LLM agents to leverage past interactions, but dialogue histories quickly exceed the context window, forcing agents to retrieve relevant subsets at query time. Because useful evidence is sparse and scattered across verbose conversations, retrieval faces a fundamental tension: broadening recall improves coverage but floods downstream reasoning with noise, while compressing memories at write time eases retrieval but irreversibly discards details that future queries may need. We introduce LazyMem, which resolves this tension by deferring all memory construction to query time. Given a retrieved candidate pool, a lightweight model processes it in overlapping parallel windows, selectively retaining and compressing only query-relevant content. The model is trained with supervised fine-tuning followed by reinforcement learning, using a reward that jointly encourages the identification of relevant messages and the generation of compressions that are faithful to the source and useful for answering the query. On LongMemEval, LazyMem-4B achieves an LLM-judge accuracy of 0.85, outperforming the strongest non-oracle baseline while using only 213 answer-context memory tokens, 21.0 times fewer than the baseline. It further generalizes to LoCoMo without target-domain training and reduces mean latency relative to the prior query-time baseline. Code is available at https://github.com/allacnobug/LazyMem.
Jing Yu, Yibo Zhao, Jiaming Zhang et al.· 1 citation
Query rewriting is a core component of web search, yet traditional methods mainly rely on large language model (LLM) prompting, fine-tuning, or personalized rewriting based on user history. These approaches often overlook population-level intent signals in large-scale query logs, leading to misalignment with mainstream search intent. Moreover, although large models achieve high rewriting quality, their computational demands and deployment complexity limit industrial applicability. To address this, we propose Population-Guided Intent-Aware Rewriting (PGIR), which captures dominant population intent via a Semantic Clustering Unit (SCU) and generates intent-aware rewrites through a Rewriting Unit (RU), aligning queries with mainstream search goals without requiring user history.Building upon PGIR, we further introduce PGIR-DPA, a dual-phase adaptation strategy that transfers capabilities from a teacher LLM to lightweight student models, achieving high-quality rewriting while ensuring industrial scalability. Extensive offline and online experiments, including A/B testing on Baidu Search, show substantial improvements in rewrite quality and a 6.28% relative increase in user satisfaction. The framework has been fully deployed in Baidu's production search system, operating stably at scale, validating its industrial feasibility and commercial value.
Yuanzhao Guo, Shuai Zhang, Ang Li et al.· Annual International ACM SIG...· 0 citations