Sep 2026· Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing· Vol abs/2609.33417· 0 citations· 37 references
Computer Science
TL;DR
This work proposes MemSuit, a self-distillation framework in which a teacher summarizer, conditioned on observed query-answer pairs, produces utility-aware memory entries that a student learns to reproduce from the raw conversation alone, confirming the value of grounding memory in downstream utility.
Abstract
Existing summarizers for memory systems are typically optimized for human-facing criteria such as faithfulness, which misaligns with their true objective: preserving the evidence needed to support future queries. We show that conditioning summarization on query-answer pairs substantially improves answer quality, and that this utility-aware behavior is transferable across queries. Motivated by these findings, we propose MemSuit, a self-distillation framework in which a teacher summarizer, conditioned on observed query-answer pairs, produces utility-aware memory entries that a student learns to reproduce from the raw conversation alone. To prevent collateral erasure where conditioning on a single query-answer pair discards evidence relevant to other plausible queries, the teacher decomposes each block into multiple self-contained entries that preserve distinct query-relevant facets as independently retrievable units. To align the retriever with the compact, fact-dense style of teacher entries, we further fine-tune the embedding model with a contrastive objective supervised by teacher entries. Across a diverse suite of conversational query types, MemSuit consistently outperforms state-of-the-art baselines, confirming the value of grounding memory in downstream utility.
This work proposes a novel training-free method that generates a very large number of queries with minimal LLM usage by compositionally combining a small set of atomic components and applies LLM reasoning to construct a high-quality document set that balances precision and recall while capturing the user's core intent.
Yunah Jang, Kang-il Lee, Joongbo Shin et al.· Conference on Empirical Meth...· 0 citations
The results show that explicitly selecting and integrating evidence improves long-context reasoning while enabling more compact generation, and the strongest overall result among the evaluated open-source models is obtained.
This work introduces DocHop, a benchmark for integrated chart--context reasoning in document-style images and constructs DocHop via a stochastic logic-first generation pipeline with controllable reasoning depth and visual density, to enable systematic evaluation.
Zhuo-Ran Yu, Le Thien Phuc Nguyen, Jaden Park et al.· 1 citation
This work introduces AgenticInterleave, a single-agent ReAct framework for retrieval-supported answer generation, together with IVR-12, a 12-dimensional rubric for assessing the content, presentation, and image quality of interleaved references and model outputs, and evaluates both short-answer correctness and interlea...
Large language models (LLMs) are increas- ingly deployed as long-horizon conversational agents, motivating growing interest in mem- ory systems. However, existing benchmarks primarily evaluate memory through QA-style probing rather than in-situ conversational usage. We introduce LOCOMO-CONV, a conversa- tional memory b...
Wen-Yu Chang, Yun-Nung Chen· 3 citations· ⚡1
Related blog posts
MIT News · Artificial Intelligence· news.mit.eduNov 19, 2026
An MIT and Dana-Farber Institute-led team used large language models to examine nearly 4,000 studies and hundreds of code repositories, revealing gaps in the information needed to scrutinize and reuse published research.
By studying the computational foundations of language learning, Associate Professor Jacob Andreas builds intelligent systems that can communicate effectively with humans.