This study introduces MVSCBench, the benchmark designed to evaluate multimodal AI models’ ability to interpret symbolic and cultural meanings in music videos. Using K-pop music video clips, we organize music video understanding into three stages: surface perception, symbolic interpretation, and cultural grounding. Based on this framework, we construct a dataset of 293 video clips and 3,516 annotations, covering space, artist, character symbolism, object symbolism, literary symbolism, social-historical symbolism, and fandom cultural symbols. The results show that current multimodal AI models perform well on surface perception tasks, but still face clear limitations in symbolic interpretation and cultural grounding. In particular, the models often struggle to correctly identify relevant visual cues and frequently produce hallucinated interpretations in categories involving deeper meanings. MVSCBench provides a new benchmark for evaluating symbolic and cultural understanding in music videos and offers a useful framework for future research.
Results show that although current models achieve strong semantic adherence and visual quality, they often fail to faithfully capture fine-grained cultural details, particularly for underrepresented regions, rituals, and multimodal cultural cues.
Xian-Jing Han, Yuhan Su, Yang Deng et al.· 2 citations
An overview of the ImageEval 2026 shared task on culturally grounded Arabic multimodal evaluation is presented, covering spoken visual question answering and image-grounded hallucination detection in English and Modern Standard Arabic, and CRAI-Bench, evaluating the cultural accuracy of text-to-image generation.
PUMA (Polish Unified Multimodal Assessment) is proposed, a novel benchmark of 900 hand-crafted tasks designed to probe the limits of multimodal models in the Polish cultural and linguistic context and open-source the evaluation framework to advance localized multimodal AI research.
Slawomir Dadas, Michał Perełkiewicz, Rafal Poswiata et al.· 0 citations
Cultural understanding in video means more than recognizing what is visible; it requires grasping the symbolic and temporal significance of cultural concepts. We decompose this into three abilities: naming what a concept symbolizes, visually recognizing it on video, and locating its sub-events in time. Existing video-c...
Burak Satar, Zhixin Ma, Yu-Tong Cheng et al.· 0 citations
Abstract Traditional board games are increasingly digitized, yet many digital adaptations reduce embodied interaction and weaken cultural meaning. This study proposes a multimodal framework to revitalize Dakon as an intangible cultural heritage practice by integrating gesture-based computer vision and large language mo...
S. Suyahman, Cecep Hilman· Preservation, Digital Techno...· 0 citations
NARU, a benchmark designed to evaluate Narrative evolution and Reasoning on cultural Understanding in Japanese long-form video, is introduced, a hierarchical memory-based annotation pipeline that transforms raw video into structured event, narrative, and cultural annotations, then generates questions via task-oriented...
Yu-Heng Huang, Jian-Lang Chen, Jia-Yang Song et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
