Skip to content
Preprint

FlashBack: Knowing When to Remember in Streaming Vision-Language Models

Oct 2026 · 0 citations · 25 references
Computer Science

TL;DR

This work introduces FlashBack, a training-free framework for selective, multi-level memory in streaming vision-language models, and shows improvements on several long-horizon and memory-dependent tasks while largely preserving real-time perception.

Abstract

Streaming vision-language models must process continuously growing video streams under a bounded compute budget, creating a persistent tension between real-time perception and long-term memory. Retrieving historical information provides a natural remedy, yet historical recall is not uniformly beneficial: unnecessary history may introduce irrelevant context into current reasoning and interfere with native real-time perception. Effective streaming memory should therefore address not only what to remember, but also when and how to access it. To this end, we introduce FlashBack, a training-free framework for selective, multi-level memory in streaming vision-language models. Before retrieving history, FlashBack draws on the semantic understanding of the frozen streaming VLM to infer whether a query calls for historical evidence. This assessment determines whether inference remains on the Native trajectory or invokes an isolated Recall trajectory. The Recall trajectory combines recent context with retrieved long-term memory through a query-local Side-KV pathway, preserving local temporal continuity without modifying the persistent Native state. We instantiate FlashBack on StreamingVLM and Mage-VL-4B and evaluate it on OVO-Bench and StreamingBench. The results show improvements on several long-horizon and memory-dependent tasks while largely preserving real-time perception, with performance competitive with strong training-based streaming methods despite requiring no additional training. Our code will be announced later.

View source

Similar papers

MemEvo: Automatic Discovery of Streaming Video Memory Mechanisms

Query-agnostic streaming video understanding requires vision-language models to continuously compress an indefinitely growing visual stream into a bounded memory before future queries are known. The performance depends critically on the memory mechanism--what observations to preserve, how to represent and consolidate t...

Guohong Liu, Jia-Lei Ye, Shan-Hui Zhao et al. · 0 citations

KeyRec: Bounded Visual Memory for Streaming and Long-Video Understanding

KeyRec is a training-free framework for constructing bounded visual memory that preserves fine-grained recent observations in a visual cache and organizes historical evidence into a structured event bank and achieves the best compressed performance in 13 of 15 settings using only 10\% of the dense decoder-facing visual...

Zi-Han Chen, Xuejian Rong, Xiao-Juan Wang et al. · 0 citations
Preprint Oct 2026

ProV-Mem: Proactive Memory for Streaming Video Question Answering

Proactive streaming video question answering requires a model to continuously monitor an incoming stream and autonomously decide when to speak, what to say, and whether to emit a response. Existing streaming systems face a fundamental difficulty: retaining long visual histories is costly, whereas discarding them can le...

Bing-Bing Wang, Jing Li, Han-Chi Gao et al. · 0 citations
#artificial intelligence Preprint Oct 2026

Memory as Weights: Internalizing Long-Term History for Streaming Videos

Streaming vision-language models (VLMs) must continuously integrate information from the past while remaining responsive to current observations and user queries. Existing memory mechanisms typically compress, retrieve, or cache historical observations and reintroduce them into the input context, forcing historical mem...

Zhuo-Ming Liu, Jin-Hong Lin, Lin Zhang et al. · 0 citations
Preprint Oct 2026

OneStreamer: Unifying Perception, Memory, and Proactive Response in Streaming Video Interaction

OneStreamer is introduced, which jointly learns query-independent evidence recording and task response through a shared proactive generation process, and the 4B model achieves the best results among the compared methods across all eight evaluated streaming video understanding benchmarks.

Xiang-Yun Zeng, Yuan-Dong Yang, Zhi-Qiu Zhang et al. · 11 citations · ⚡2
Preprint Sep 2026

ContextTom: Training-Free Video VLM Acceleration with Early Spatial Context

Video vision-language models process thousands of visual tokens before producing an answer. Reducing these tokens early saves computation, but also determines which evidence reaches the language model. We introduce ContextTom, a training-free framework that separates the timing of temporal and spatial compression. Its...

Yu-Qi Li, Neilier Rodrigues, Xiang-Long Wang et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.