Aug 2026· Engineering Research Express· Vol 8, pp. 175209· 0 citations· 37 references
Physics
TL;DR
BiM-MFGCN, a bidirectional Mamba and multi-frequency graph convolutional network for multimodal ERC, is proposed, demonstrating its effectiveness for multimodal ERC.
Abstract
Emotion recognition in conversation (ERC) automatically identifies the emotional states conveyed by individual utterances in multi-turn dialogues and is valuable for intelligent customer service, mental-health monitoring, and human-computer interaction. Existing multimodal ERC methods still face challenges in capturing bidirectional contextual dependencies, preserving local emotional variations, and integrating complementary cross-modal cues. To address these issues, we propose BiM-MFGCN, a bidirectional Mamba and multi-frequency graph convolutional network for multimodal ERC. The model contains a bidirectional Mamba temporal modeling module (BAM) and a multi-frequency graph attention module (FGA). BAM employs forward and backward Mamba2 paths to encode preceding and subsequent dialogue context. FGA constructs an adaptive composite utterance graph, separates low-, mid-, and high-frequency graph-spectral components, and performs cross-modal interaction on frequency-enhanced representations. Experiments on MELD and IEMOCAP show that BiM-MFGCN achieves 66.66% accuracy and 67.82% weighted F1 on MELD, and 70.11% accuracy and 70.12% weighted F1 on IEMOCAP, demonstrating its effectiveness for multimodal ERC.
Multimodal Emotion Recognition in Conversation (MERC) aims to identify speakers' emotions by integrating conversational context with textual, acoustic, and visual cues. Existing methods remain limited in capturing emotion-relevant contextual dependencies and modeling cross-modal discrepancies during multimodal fusion....
Yang-Tian She, Yu-Ming Lin, Ling-Zhen Wu et al.· Proceedings of the 34th ACM...· 0 citations
Multimodal emotion recognition in conversations aims to identify the emotion expressed in each utterance by jointly analyzing textual, acoustic, visual, and contextual information. However, existing methods still face challenges in coordinating intra-modal dependency modeling, cross-modal interaction, and graph-based c...
Jing Zhao, Feng Liu· Frontiers in Psychology· 1 citation
Multimodal emotion recognition in conversations (MERC) aims to identify utterance-level emotions from textual, acoustic, and visual cues, and provides a basis for future extension to richer multisensory affective understanding. However, semantic inconsistency across heterogeneous modalities, the under-utilization of mo...
Luntian Mou, Yue Zeng, Sha-Sha Mao et al.· Proceedings of the 2nd Inter...· 0 citations
Psychological stress dialogue recognition still lacks sufficient semantic understanding in multimodal information fusion and emotional dynamic modelling. It is difficult to achieve high-precision and robust stress state recognition in complex dialogue scenarios. Combining the advantages of the pre-trained language mode...
The Discourse-Guided Multi-View Adaptive Fusion framework constructs multiple discourse-guided views and constrains information propagation within each view through discourse-guided masking, thereby reducing irrelevant contextual interference and better preserving fine-grained local emotional cues.
You Li, Yang-Tian She, Li-Hua He et al.· World wide web (Bussum)· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.