Skip to content

Exact Record Omission in Delta Attention: A Transport Criterion, Its Cost, and a Replay Certificate

Sep 2026 · 2 citations · 30 references
Computer Science

TL;DR

These results provide a practical audit of both the requested memory change and the assistant's remaining behavior, with replay work determined by the surviving suffix.

Abstract

An assistant can stop repeating a deleted statement while its recurrent memory still carries that statement's influence. We examine this distinction by saving the state difference immediately after a record, carrying this saved difference, or receipt, through later updates, and comparing the corrected state with the state built from the same conversation with the record omitted. Unrolling the recurrence gives an exact criterion: transport reaches this never-stored state if and only if the additional differences created by later updates cancel after transport. We evaluate eleven conditions using 40 prospectively selected synthetic Kimi Linear contexts, with 40 separate calibration contexts and paired uncertainty estimates computed across complete records. Attention masking removed all 40 exact greedy target answers, yet a fixed three-query candidate-scoring attack with known candidate values and log-probability access attained AUC 0.740 (95% interval 0.695--0.809); the measured false-positive rate was 7.5%. Prompt-only forgetting still returned 33 of the 40 targets. Checkpoint replay matched the complete declared active state and audit logits in all 80 contexts, while preserving all 120 retained answers in the evaluation cohort. An independent Qwen cohort and matched original-bf16/8-bit Kimi controls confirmed native transport mismatch beyond the numerical error measured in matched controls. Replay also supported record replacement and successive deletions with a new record added between them. These results provide a practical audit of both the requested memory change and the assistant's remaining behavior, with replay work determined by the surviving suffix.

View source

Similar papers

#artificial intelligence Preprint Sep 2026

Budget Boundary Effects in Test-Time Mathematical Reasoning

A cumulative token cap can fall inside a mathematical derivation, forcing a test-time controller to choose between stopping at the cap (strict) and allowing the current attempt to finish (advisory). We measure this boundary choice with paired offline replays of 19,200 public traces: 120 AIME, BrUMO and HMMT problems an...

Gui-Lin Zhang, Zi-Qi Tan, Wu-Lan Guo et al. · 0 citations
#machine learning Preprint Sep 2026

Exact Finite Attention Responses From RoPE Derivatives

We derive exact local responses for attention interventions, allowing candidate edits to be scored from a cached baseline and one backward pass. The starting point is the RoPE derivative $\partial_p z(p) = A z(p)$: its integral gives the finite positional displacement, which we carry through the softmax without lineari...

Julie Huang, Maggie Chlon, G. Gutin et al. · 0 citations
#artificial intelligence Preprint Sep 2026

LAM: Efficient Lossy Agent Memory Framework With A Retrieval-Score Error Bound

LAM is proposed, a Lossy Agent Memory system with three components: a deterministic deduplication rule with a substitution bound on retrieval scores - a bound on score perturbation, not a certificate of unchanged ranking; a memory manager that preserves the cached prefix and overlaps compaction with inference; and a pe...

Bai-Xi Sun, Le Chen, Anjir Ahmed Chowdhury et al. · 0 citations
Preprint Aug 2026

L\'evy Attention: Single-Pass Predictive Uncertainty for Continuous-Time Attention

This work shows the attention layer itself can close that gap between deep models for irregularly-sampled time series answer queries at arbitrary continuous timestamps: with the right stochastic formulation, the pass that makes each prediction also reports how far it should be trusted.

S. Chatzis, Loukas Papadoulas · 0 citations

Related blog posts

MIT News · Artificial Intelligence Aug 27, 2026

Looking beyond natural sequences

A new machine-learning framework aims to improve the success rate of computational protein design while moving away from results that reproduce sequences found in nature.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.