Skip to content

Learning to Prefer Reliably: Error-Augmented Emotion Preference Optimization with Calibrated Fusion

Aug 2026 · 0 citations · 31 references
Computer Science

TL;DR

Experiments on the MER2026-EmoPrefer Challenge dataset and the error-augmented dataset demonstrate that EAPO improves emotion preference prediction and enhances the robustness of MLLM judges when evaluating fluent descriptions that conflict with the video's multimodal emotional evidence.

Abstract

Emotion preference learning uses pairwise comparisons between candidate descriptions to align multimodal large language models (MLLMs) with human judgments of open-ended emotion descriptions and to train reward models that capture human emotional preferences. However, conventional pairwise supervision is often sparse, typically providing only a single negative description for each positive description, and therefore offers limited coverage of the diverse ways in which an emotion description can be incorrect. In particular, models may be insufficiently exposed to semantically fluent but emotionally inconsistent descriptions. Beyond this data-level limitation, relying on a single MLLM judge introduces a distinct model-level concern: its judgments can be affected by model-specific biases when interpreting fine-grained or ambiguous multimodal emotional cues. To address these limitations, we propose Error-Augmented Preference Optimization (EAPO), a framework for improving the reliability of MLLM-based emotion preference judgment at both the data and model levels. First, we construct an error-augmented dataset by generating multiple controlled and emotion-aware negative descriptions from each preferred description. We then adapt multiple independent MLLM judges to this richer supervision and aggregate their preference margins using margin-calibrated soft fusion, which maps heterogeneous margins to a common scale before aggregation. Experiments on the MER2026-EmoPrefer Challenge dataset and our error-augmented dataset demonstrate that EAPO improves emotion preference prediction and enhances the robustness of MLLM judges when evaluating fluent descriptions that conflict with the video's multimodal emotional evidence. Our code is available at https://github.com/slash1028/EAPO-EmoPrefer.

View source

Similar papers

#machine learning Preprint Oct 2026

EmoGavel: Grounding-Aware Reinforcement Learning with Verifiable Rewards for Multimodal Emotion Preference Learning

Multimodal emotion preference learning uses pairwise comparisons to identify which open-ended emotion descriptions align more closely with human judgments. However, existing methods primarily focus on the final preference decision and provide little explicit supervision for localizing and correcting erroneous content....

Zi-Long Huang, K. Lee, Ze-Zhong Jin et al. · 0 citations
Sep 2026

Decoding Affective Nuances: Enhancing MLLMs via Hierarchical Emotion Reasoning and Contrastive Discriminative Pruning

A Hierarchical Emotional Reasoning Chain (HERC) is proposed that enhances the insufficient attribution by harmonizing fine-grained scene/object-level cues and performing a soft-gated reasoning, and a Contrastive Discriminative Visual Pruning (CDVP), which isolates discriminative visual tokens to reason the final emotio...

C. Ye, Wei-Dong Chen, Zhao-Bo Qi et al. · 0 citations
Sep 2026

DSPO: Diversity-aware Subjective Policy Optimization for Robust Emotional Reasoning

DSPO, a reinforcement learning framework that jointly promotes subjective affective coverage and visual grounding, achieves state-of-the-art performance across multiple public benchmarks, especially on the cross-domain performance, i.e., improving +10.8\% on average cross-domain accuracy than EMO-R3.

C. Ye, Wei-Dong Chen, Bing-Yan Xu et al. · 0 citations

EmoICL: Enhancing Visual Emotion Recognition With Emotion-Aware In-Context Learning

Visual emotion recognition aims to infer human emotional states from visual cues. Adapting pretrained Vision–Language Models (VLMs) to this task remains challenging. In-Context Learning (ICL) enables a frozen model to condition its predictions on a few labeled demonstrations without task-specific fine-tuning. Here, dem...

Xi-Yang Li, Qu Yang, Yu-Wei Yin et al. · 0 citations
Preprint Aug 2026

LG-GER: Language-Guided Group Emotion Recognition via Multimodal Evidence Distillation

LG-GER is proposed, a language-guided distillation framework that uses a multimodal large language model (MLLM) to generate dense, spatially grounded evidence that achieves competitive or superior results compared to state-of-the-art methods that require detection and multi-stream processing at inference.

Ahmed-Shehab Khan, Zhiyuan Li, Yan Tong · 0 citations
#artificial intelligence Preprint Oct 2026

From Surface to Depth: Towards Cognitive Appraisal Reasoning in Multimodal Emotion Understanding

Recent multimodal large language models (MLLMs) increasingly incorporate explainable reasoning for emotion understanding. However, reasoning based mainly on observable affective cues can reduce emotion understanding to superficial cue-label associations, giving rise to the Clever Hans effect. Such shortcuts become unre...

Jia Li, Yi-Chao He, Yang-Chen Yu et al. · 0 citations

Related blog posts

MIT News · Artificial Intelligence Nov 17, 2026

Putting a spotlight on AI, society, and humanity

Supported by the MIT Human Insight Collaborative, scholars are exploring how AI is impacting the human experience and reshaping higher education.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.