Skip to content

Correct Now, Insufficient Later: Auditing Update Sufficiency in Context Compression

Sep 2026 · 0 citations · 18 references
Computer Science

TL;DR

A paired-history audit and label-equivariant repair support a scoped evaluation methodology and reproducible failure analysis and general superiority of the repaired algorithm, not general superiority of the repaired algorithm.

Abstract

A memory can answer a current query correctly while discarding distinctions required by a later update. We investigate this failure with a paired-history audit: two histories have the same current answer, receive a shared future update, and require different subsequent answers. A pilot evaluates 24 history pairs across six synthetic mechanisms, 12 memory conditions, two repeats, and two model backends. A deterministic frontier selector obtains strict reveal accuracy of 96/96 on DeepSeek and 82/96 on GLM; a structured writer obtains 62 successes with one unresolved outcome and 56/96. The configured four-outcome joint contrast has finite-sample identification intervals of [0.521, 0.542] and [0.292, 0.313], not confidence intervals. A record-level audit distinguishes retained-state adequacy, response delivery, and answer-schema compliance without changing those original scores. It finds 26 and 25 well-formed but semantically wrong structured reveal memories, while all 14 GLM frontier reveal failures contain correct values in the wrong wrapper. Tombstone removal produces 16/16 exact replay failures in the targeted mechanism. Identifier renaming then exposes a separate flaw: original frontier late-reference adequacy falls from 8/8 to 94/320 transformed instances. We provide and test a label-equivariant repair, but it preserves only 2/8 original late-reference answers: eliminating a naming shortcut does not solve unknown future relevance. These results support a scoped evaluation methodology and reproducible failure analysis, not general superiority of the repaired algorithm. Paid pilot evidence, retrospective diagnostics, and new offline tests are reported separately; no independent held-out or natural-task validation is claimed.

View source

Similar papers

Preprint Sep 2026

A Historical Corpus Is Not a Historical System: Auditing Hindsight Leakage in Stateful Data Discovery

Offline replay should estimate what a discovery system could retrieve at a historical point, yet freezing the corpus leaves interaction memory unconstrained. We formalize point-in-time (PIT) discovery through historical state $(D_t, \theta_t, M_{<i})$ and introduce a paired replay that changes only memory availability....

Yi-Xi Zhou, Fan Zhang, Si-Kun Wang et al. · 0 citations
#artificial intelligence Preprint Oct 2026

The Right Memory in the Wrong Context: Verifying Retrieval Admissibility in Long-Term Agent Memory

Long-term-memory agents can retrieve relevant information that is inadmissible for the current request because it belongs to another principal, violates policy, or reflects an incompatible lifecycle state. Recall and final-answer accuracy do not reveal this: a route can appear safe by missing required evidence, while a...

Zi Wang, Xing-Qiao Wang, Emmanuel Addai et al. · 0 citations
Preprint Aug 2026

Explicit State Elicitation Is Not Enough: A Controlled Audit of Memory-Policy Classification

Family-level and seed-stability analyses show that example-level accuracy overstates counterfactual consistency: complete four-way family success is rare, and an exploratory follow-up that elicits decomposed semantic evidence fails to improve routing for the cleanly evaluated endpoint.

Yi-Hang Chen, Pinyan Qian, Su Wang et al. · 0 citations
#artificial intelligence Preprint Sep 2026

LSREP: A Longitudinal State-Replay Protocol for Evaluating Conversational Memory, with ICE v2 as an Audited Local-First Architecture

Conversational memory changes during use, so endpoint question answering alone cannot establish how a persistent state accumulates, ages, or incorporates revisions. We introduce LSREP, a Longitudinal State-Replay Evaluation Protocol combining ordered replay, explicit lifecycle schedules, repeated probes, evolving refer...

Deep K. Sonar · 0 citations
#artificial intelligence Preprint Sep 2026

What Should an Agent Remember? Disentangling Retention from Retrieval in Bounded-Memory Evaluation

A persistent agent must decide both what to retain as information arrives and what to surface once a query appears, yet memory evaluations can confound these decisions by comparing methods that differ in both retention and selection. We build a streaming-recall benchmark crossing retention and selection rules and evalu...

Ju-Li Huang · 0 citations

Related blog posts

MIT News · Artificial Intelligence Sep 29, 2026

Who we become when we talk to machines

Professor Sherry Turkle’s new book, “Artificial Intimacy,” offers a withering critique of chatbots and the antisocial dynamics she believes they encourage.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.