When AI translations fall flat: embodied simulation and effort in cross-language reading

Abstract

Common machine translation evaluation metrics, such as the Bilingual Evaluation Understudy (BLEU) and Crosslingual Optimized Metric for Evaluation of Translation (COMET) are reliable measures of propositional fidelity and surface fluency but systematically ignore the embodied texture of reading, leaving unexplained why high-scoring AI translations still feel flat to readers despite formal accuracy. The Embodied Translation Quality Assessment (E-TQA) framework was developed based on embodied cognition theory to operationalize three key dimensions of cross-language comprehension: sensorimotor simulation, affective-cultural resonance, and processing friction. To validate this framework, a counterbalanced Latin-square design was employed with 192 English-major participants. Six literary and culturally loaded excerpts were each presented in three rigorously matched conditions. Published professional human translations, and translations from DeepSeek and ChatGPT, with all versions controlled for BLEU (±1.2 points) and COMET (±0.04 points) to isolate embodied variation from semantic equivalence. Participants completed the Self-Assessment Manikin and E-TQA scales after each excerpt, and data were analyzed via linear mixed-effects models, reporting partial η 2 as effect size. Human texts were consistently rated higher than both AI conditions in sensorimotor imagery, affective resonance, valence, and arousal, whereas AI texts were rated much higher in phenomenological flatness and cognitive effort. All critical indices showed medium to large effect sizes (η p 2 = 0.15–0.35) and were consistent across educational levels and text genres. These results offer evidence for a dissociation of propositional accuracy and embodied richness, showing that the E-TQA framework accounts for unique variance in reader experience beyond that explained by standard computational metrics. The work advances translation quality assessment by centering embodied cognition, provides a psychometrically sound complement to existing tools, and motivates future multimodal investigations of cross-language processing mechanisms.