ACERT (Averaged Contextual Emotion Representation through Time), a module that integrates a flexible-length window of conversational context to better capture emotional evolution in spoken interactions, is introduced.
Abstract
Detecting emotions is necessary for building systems that can accurately and adaptively interact with humans. Speech Emotion Recognition (SER) has become an important research focus to develop intelligent spoken interfaces. However, most studies predict emotions at the utterance level, ignoring the conversational context, along with the emotional flow and speaker interactions it carries. In this paper, we introduce ACERT (Averaged Contextual Emotion Representation through Time), a module that integrates a flexible-length window of conversational context to better capture emotional evolution in spoken interactions. To evaluate the robustness of this method, we conducted experiments on datasets spanning diverse emotionally expressive styles and contexts. ACERT outperforms current state-of-the-art (SOTA) approaches on IEMOCAP, establishes the first context-aware benchmark on SAFE, and obtains strong results on MELD for unweighted, class-balanced metrics. Ablation studies show that ACERT's gains come from emotional and conversational continuity, rather than from speaker identity or acoustic conditions.
An LLM-as-Judge framework is introduced that evaluates each emotion independently according to its plausibility in the conversational context rather than enforcing a single-label decision, suggesting that standard single-label evaluation is therefore insufficient.
Amir Ben Khalifa, Fanny Bezancon, B. Abdulrazak et al.· 0 citations
Speech emotion recognition (SER) is the task of assigning emotion labels to utterances. Early systems relied on acoustic features, whereas recent approaches combine multiple modalities, most commonly speech and text. Still, performance remains poor on many datasets. Large language models (LLMs) have therefore attracted...
He-Zhao Zhang, Thomas Hain· arXiv.org· 0 citations
DSSM-CRF, an audio-only architecture that explicitly separates cross-speaker contextual influence and within-speaker emotion evolution, is proposed and matched controls demonstrate complementary gains from speaker-wise factorization and CRF modeling.
This study forms continuous recognition of the Group Emotion at a one-second resolution, introduces the Mixed state, which captures the emotional divergence among participants in the group, and proposes a multimodal temporal framework that integrates audio and video information using a sliding-window context.
Soma Iwata, K. Inoue, Mu-Yun Wu et al.· Companion Publication of the...· 0 citations
This work reformulates AER as a Temporal Affective Grounding task that associates emotions with temporally bounded speech spans and vocal tone descriptions, and introduces Masked Temporal Affective Grounding (M-TAG), a supervised training objective that combines full-sequence language modeling with emotion and timestam...
Abdel-rahman Mohamed, Lars Kai Hansen, Zheng-Hua Tan· arXiv.org· 0 citations
An LLM-based framework that performs discrete emotion recognition and Valence-Arousal-Dominance dimensional evaluation on IEMOCAP, incorporating acoustic cues as natural language descriptions following the SpeechCueLLM approach is proposed.
Yu-Tong Hu, Jin-Ho D. Choi· arXiv.org· 0 citations
Related blog posts
MIT News · Artificial Intelligence· news.mit.eduNov 20, 2026
Senior Research Scientist Jim Glass discusses how his MIT-IBM Computing Research Lab collaboration connects perception and language, and addresses the shortcoming of current models.
MIT News · Artificial Intelligence· news.mit.eduNov 19, 2026
An MIT and Dana-Farber Institute-led team used large language models to examine nearly 4,000 studies and hundreds of code repositories, revealing gaps in the information needed to scrutinize and reuse published research.
By studying the computational foundations of language learning, Associate Professor Jacob Andreas builds intelligent systems that can communicate effectively with humans.
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.