Skip to content
#small language model Book Open access

MERIT: Dynamics-Aware Visual Token Compression for High-Resolution Document Understanding in Multimodal Large Language Models

Nov 2026 · Proceedings of the 34th ACM International Conference on Multimedia · 0 citations · 8 references

TL;DR

The proposed MERIT framework, a training-free visual token compression framework that estimates token importance through multi-view dynamics from global semantics, local structure, and signal saliency, and Dynamics-Aware Proxy Compression (DAPC), which preserves high-value tokens via feature-faithful hard sampling and compensates for the influence of compressed regions in subsequent attention through quality redistribution are developed.

Abstract

High-resolution visual inputs can substantially enhance the fine-grained understanding of documents, charts, and complex layouts in multimodal large language models (MLLMs), but they also incur higher inference latency and greater memory overhead. Existing visual token compression methods primarily rely on pruning or merging strategies, yet they remain inadequate for document-centric scenarios: the former may erroneously discard low-saliency but semantically critical regions, such as small-font text and symbols, while the latter may attenuate the high-frequency local details essential for OCR through feature averaging.To address these limitations, we propose MERIT, a training-free visual token compression framework that estimates token importance through multi-view dynamics from global semantics, local structure, and signal saliency. Building on this design, we further develop Dynamics-Aware Proxy Compression (DAPC), which preserves high-value tokens via feature-faithful hard sampling and compensates for the influence of compressed regions in subsequent attention through quality redistribution. Experimental results demonstrate that MERIT achieves strong performance across document understanding, OCR, and general multimodal benchmarks. On LLaVA-OneVision-7B, MERIT retains 97.6% and 93.0% of the original performance at compression ratios of 66.7% and 77.8%, respectively, while reducing time-to-first-token (TTFT) by 62.9% and 60.7%, respectively, relative to the best competing baselines under the respective settings. These results suggest that dynamics-aware, feature-faithful compression provides an effective pathway toward efficient high-resolution multimodal document understanding.

Read PDF

Similar papers

FocusVTC: Efficient and High-Performance Visual Text Compression with Adaptive Resolution

FocusVTC, which combines compressed low-DPI global views with selective region enhancement, integrating enhanced views into ongoing reasoning, is introduced, which breaks this trade-off through adaptive resolution while preserving general multimodal capabilities.

Fang-Zhi Zhong, Xue-Rui Qiu, Yu-Qi Pan et al. · 1 citation
Preprint Oct 2026

Don't Ask the LLM: Visual Token Pruning with Spectral Structure and Semantic Relevance

Large vision-language models (LVLMs) represent images and videos as long visual-token sequences, substantially increasing inference costs. These costs motivate visual-token pruning, whose effectiveness depends on both what signal is used for selection and when that signal becomes available. Our analysis reveals complem...

Intaek Shin, Geuntaek Lim, Sungjune Park et al. · 0 citations
Book Open access Nov 2026

SG-MoE: Efficient Visual Compression for Multimodal Large Language Models via Semantic-Guided Mixture-of-Experts

With the rapid advancement of Multimodal Large Language Models (MLLMs), the redundancy of visual tokens in images has become a critical bottleneck constraining inference efficiency. Existing resampling methods often suffer from detail loss and fail to dynamically adapt to user instructions. In this paper, we propose SG...

Zhan-Pei Zhang, Yu Gao, Heng Li et al. · 0 citations
Preprint Oct 2026

Hierarchical and Query-Aware Visual Token Pruning for High-Resolution LVLMs

This work proposes a training-free, hierarchical token pruning framework that utilizes query guidance prior to executing any LLM blocks, and demonstrates that this plug-and-play framework retains 98.0% and 94.8% of unpruned performance at 25% and 10% vision token retention, respectively.

Luther Wei, Ling Zou, Kang-Yang Huang et al. · 0 citations
Book Open access Nov 2026

DATP: Training-Free Depth-Aware Token Pruning for Vision-Language Models

The massive redundancy of visual tokens in Vision-Language Models (VLMs) creates a critical computational bottleneck during large language model (LLM) prefill. Recent training-free pruning methods provide plug-and-play acceleration, yet their pruning decisions often rely on limited visual states or delayed in-LLM signa...

Qing-Xin Liu, Jin-Zhi Liao, Xiang Zhao · 0 citations
Preprint Oct 2026

Adaptive Visual Token Reduction for Accelerated Image Understanding

Large Vision-Language Models achieve strong VQA performance, but processing high-resolution, information-rich images requires substantial computation, motivating visual token reduction. However, existing methods often prune individual tokens or rely on fixed-size cropping, limiting their ability to preserve spatially s...

Seyoung Jeong, J. Yun, Sang-Jun Lee · 0 citations

Related blog posts

Google DeepMind Blog Nov 18, 2026

Genie 4 is now available

We are rolling out a new Project Genie experience to eligible Google AI Ultra subscribers, opening access to our new Genie 4 world model.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.