Nov 2026· Proceedings of the 34th ACM International Conference on Multimedia· 0 citations· 8 references
TL;DR
The proposed MERIT framework, a training-free visual token compression framework that estimates token importance through multi-view dynamics from global semantics, local structure, and signal saliency, and Dynamics-Aware Proxy Compression (DAPC), which preserves high-value tokens via feature-faithful hard sampling and compensates for the influence of compressed regions in subsequent attention through quality redistribution are developed.
Abstract
High-resolution visual inputs can substantially enhance the fine-grained understanding of documents, charts, and complex layouts in multimodal large language models (MLLMs), but they also incur higher inference latency and greater memory overhead. Existing visual token compression methods primarily rely on pruning or merging strategies, yet they remain inadequate for document-centric scenarios: the former may erroneously discard low-saliency but semantically critical regions, such as small-font text and symbols, while the latter may attenuate the high-frequency local details essential for OCR through feature averaging.To address these limitations, we propose MERIT, a training-free visual token compression framework that estimates token importance through multi-view dynamics from global semantics, local structure, and signal saliency. Building on this design, we further develop Dynamics-Aware Proxy Compression (DAPC), which preserves high-value tokens via feature-faithful hard sampling and compensates for the influence of compressed regions in subsequent attention through quality redistribution. Experimental results demonstrate that MERIT achieves strong performance across document understanding, OCR, and general multimodal benchmarks. On LLaVA-OneVision-7B, MERIT retains 97.6% and 93.0% of the original performance at compression ratios of 66.7% and 77.8%, respectively, while reducing time-to-first-token (TTFT) by 62.9% and 60.7%, respectively, relative to the best competing baselines under the respective settings. These results suggest that dynamics-aware, feature-faithful compression provides an effective pathway toward efficient high-resolution multimodal document understanding.
FocusVTC, which combines compressed low-DPI global views with selective region enhancement, integrating enhanced views into ongoing reasoning, is introduced, which breaks this trade-off through adaptive resolution while preserving general multimodal capabilities.
Fang-Zhi Zhong, Xue-Rui Qiu, Yu-Qi Pan et al.· arXiv.org· 1 citation
Large vision-language models (LVLMs) represent images and videos as long visual-token sequences, substantially increasing inference costs. These costs motivate visual-token pruning, whose effectiveness depends on both what signal is used for selection and when that signal becomes available. Our analysis reveals complem...
Intaek Shin, Geuntaek Lim, Sungjune Park et al.· 0 citations
With the rapid advancement of Multimodal Large Language Models (MLLMs), the redundancy of visual tokens in images has become a critical bottleneck constraining inference efficiency. Existing resampling methods often suffer from detail loss and fail to dynamically adapt to user instructions. In this paper, we propose SG...
Zhan-Pei Zhang, Yu Gao, Heng Li et al.· Proceedings of the 34th ACM...· 0 citations
This work proposes a training-free, hierarchical token pruning framework that utilizes query guidance prior to executing any LLM blocks, and demonstrates that this plug-and-play framework retains 98.0% and 94.8% of unpruned performance at 25% and 10% vision token retention, respectively.
Luther Wei, Ling Zou, Kang-Yang Huang et al.· 0 citations
The massive redundancy of visual tokens in Vision-Language Models (VLMs) creates a critical computational bottleneck during large language model (LLM) prefill. Recent training-free pruning methods provide plug-and-play acceleration, yet their pruning decisions often rely on limited visual states or delayed in-LLM signa...
Qing-Xin Liu, Jin-Zhi Liao, Xiang Zhao· Proceedings of the 35th ACM...· 0 citations
Large Vision-Language Models achieve strong VQA performance, but processing high-resolution, information-rich images requires substantial computation, motivating visual token reduction. However, existing methods often prune individual tokens or rely on fixed-size cropping, limiting their ability to preserve spatially s...
An MIT and Dana-Farber Institute-led team used large language models to examine nearly 4,000 studies and hundreds of code repositories, revealing gaps in the information needed to scrutinize and reuse published research.