Vision-language models (VLMs) have demonstrated excellent performance in visual tasks, but their visual causal reasoning capabilities still lack reliable evaluation. Existing evaluations struggle to distinguish whether a model is performing causal reasoning based on visual evidence or relying on statistical correlation...
Direct-decision models enable efficient classification and evaluation by selecting from predefined candidates, but their reliability depends on more than prediction accuracy. Across 40 datasets and nine models from the JEV, KEV, and StartLux Decision families, we identify a systematic tendency to underuse ordinal ratin...
Tian-Xiang Gao, Jin-Zhe Li, Zhiyuan Li et al.· 3 citations
These results show that reliable scene-text recognition requires VLMs to balance visual character evidence with contextual information, preserving clear text while using context mainly when the visual evidence is uncertain.
Findings show that terminal failure alone establishes neither behavioral disengagement nor refusal and that outcomes alone are insufficient for VLA evaluation, and that outcomes alone are insufficient for VLA evaluation.
It is found that competence dominates human-directed evaluations, while many \textit{other} attributions describe humans as epistemic, cultural, or embodied subjects, and suggest that bias in agent societies should be studied not only as isolated model output, but also as a discourse process.
Overall, assistant-generated history has active but selective effects on multi-turn performance, motivating selective rather than uniform history management.
Jinnan Li, Zheren Fu, Yue Wang et al.· 0 citations
VeriOCRBench, a 1,800-sample human-verified benchmark built from source images drawn from 8 OCR-related datasets and spanning 8 real-world image domains, with controlled, image-grounded diagnostic tasks, is introduced, exposing a critical reliability gap in current OCR reasoning systems.
RPCBench, a benchmark for evaluating Recommender-Premise Critique: the ability to detect, diagnose, and properly handle faulty premises in natural-language recommendation requests, is introduced and a fine-grained evaluation framework that measures proactive detection, error localization, post-detection handling strate...
This work proposes MMPCBench, a comprehensive framework for evaluating MLLMs' proactive critique competence, which adopts a hierarchical evaluation protocol to measure models' error detection, diagnosis and resolution performance, and applies alignment-aware metrics to assess the coherence between internal reasoning an...
Jin-Zhe Li, Geng-Xu Li, Jinnan Li et al.· 0 citations
Experiments with representative closed-source and open-source MLLMs show that OCR-grounded meta-reasoning remains far from saturated: models struggle with visible-rule application and layout-sensitive inference, while process-compliant rationales can accompany incorrect final answers under exact-match evaluation.
By eliminating routers entirely, CD-LoRA employs a consistency-driven alignment mechanism to enforce representation congruence across tasks in a shared low-rank space, which fosters robust, task-agnostic features without explicit partitioning overhead.
CaSKG, a counterfactual-causal skill graph framework that calibrates procedural relations before retrieval, is proposed, position edge-confidence calibration as an effective route to compact and executable skill retrieval at scale.
Zhiyuan Li, Lin-Yuan Gao, Xue-Chun Ding et al.· 3 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
