Mixture of Cognitive Experts in Large Vision-Language Models

Seeing Helps Reasoning in Language Models

Seeing or Knowing? Visual Context Sensitivity in Multimodal Large Language Models

For the coarse attributes the authors study, MLLMs encode the visual evidence but cannot reliably control their reliance on it, indicating that for the coarse attributes they study, MLLMs cannot reliably control their reliance on it.

Enhancing Visual Reasoning via Structure-Aware Learning in Vision–Language Models

Vision-Language Models (VLMs) have achieved remarkable progress in aligning visual and textual information; however, their inference processes remain largely implicit, end-to-end, and weakly structured. As a result, even state-of-the-art models often struggle with logical consistency, spatial reasoning, multi-entity binding, and robustness to occlusion or viewpoint changes, limiting their reliability in scientific, industrial, and safety-critical applications. To address these limitations, we propose Structure-Aware Visual Reasoning (SAVR), a conceptual framework that augments VLMs with explicit representation and constraint-based reasoning components. SAVR decomposes visual inference into three interpretable stages: (i) an entity–attribute–relation (EAR) representation, (ii) constraint-aware reasoning over structured representations, and (iii) integration of structured predictions with VLM-generated outputs. This formulation treats visual reasoning as an explicit process that enforces spatial, logical, and physical constraints over grounded entities, thereby improving consistency, transparency, and controllability. Furthermore, SAVR unifies several previously fragmented research directions-including scene graphs, object-centric learning, neuro-symbolic reasoning, and spatially grounded VLMs-into a coherent architectural perspective. Through this synthesis, we clarify how explicit structural representations can systematically mitigate common failure modes of end-to-end VLMs. While this work is conceptual and does not include empirical evaluation, it provides rigorous problem formulation, design rationale, and comparative analysis that establish a foundation for future structure-aware multimodal reasoning systems.

E2S-Pruner: Progressive Two-Stage Evidence Fusion for Visual Token Pruning in Vision-Language Models

Vision-language models typically encode an image into hundreds of visual tokens, incurring substantial inference latency and GPU memory overhead. Existing pruning methods largely rely on attention scores and directly aggregate outputs across attention heads and network layers, making it difficult to characterize evidential uncertainty and conflict. We propose E2S-Pruner, a progressive two-stage evidence-fusion framework for visual token pruning that requires no auxiliary model, trainable parameters, or fine-tuning. In the first stage, E2S-Pruner treats each attention head as an independent evidence source, estimates its reliability from evidence clarity and inter-head consistency, and represents each visual token using three states: important, unimportant, and uncertain. In the second stage, Dempster--Shafer evidence theory is used to quantify inter-layer conflict and fuse complementary evidence from multiple network layers. We further introduce a spatial novelty constraint that promotes coverage of distinct image regions and prevents the retained tokens from concentrating in a few locally salient areas. On LLaVA-1.5-7B, E2S-Pruner retains 98.0%, 96.8%, and 90.6% of the aggregate performance when the average numbers of retained visual tokens are 192, 128, and 64, respectively, while improving throughput by 1.96x and 2.09x under the 128-token and 64-token settings. Experiments on Qwen2-VL-7B further demonstrate cross-model generalization. Code is available at https://github.com/taoyu-qian/E2S-Pruner.git.

UniProbe: A Learnable Token-Level Hallucination Detector for Large VLMs using Multi-Structural Internal Representations

UniProbe is introduced, a lightweight, unified, learnable detector that models a frozen LVLM's heterogeneous computational trace from a single forward pass and achieves state-of-the-art token-level and object-hallucination detection.

GLaQ: Grounding Latent Queries in Visual Evidence for Multimodal Reasoning

Chain-of-thought reasoning has substantially improved the problem-solving capabilities of multimodal large language models. Fine-grained visual evidence, however, remains difficult to preserve and reuse across text-based reasoning steps. To address this limitation, tool-augmented thinking-with-images methods maintain visual access externally by revisiting or manipulating the image, but require predefined tools and additional inference-time processing. As an internal alternative, continuous visual latent reasoning retains intermediate computation in hidden states. However, its prevailing autoregressive construction makes each latent state depend on its predecessors, so later states may repeat information already present in the latent sequence rather than capture complementary visual details. We introduce GLaQ, a grounded latent-query framework that replaces sequential latent rollout with a fixed set of context-conditioned queries grounded in the original visual tokens. The grounded queries are reinjected for answer generation, providing direct and coordinated access to source visual evidence. We train GLaQ with localized-view supervision followed by reinforcement learning under task-level rewards. Across five benchmarks for fine-grained visual understanding and perception, GLaQ-7B gains 5.99--9.66\% over its base model and leads all compared visual latent methods, suggesting that direct query-to-image grounding can recover localized evidence from the full image without external visual operations or autoregressive latent rollouts.