Skip to content
Open access

BiM-MFGCN: a bidirectional Mamba and multi-frequency graph convolutional network for multimodal emotion recognition in conversation

Aug 2026 · Engineering Research Express · Vol 8, pp. 175209 · 0 citations · 37 references
Physics

TL;DR

BiM-MFGCN, a bidirectional Mamba and multi-frequency graph convolutional network for multimodal ERC, is proposed, demonstrating its effectiveness for multimodal ERC.

Abstract

Emotion recognition in conversation (ERC) automatically identifies the emotional states conveyed by individual utterances in multi-turn dialogues and is valuable for intelligent customer service, mental-health monitoring, and human-computer interaction. Existing multimodal ERC methods still face challenges in capturing bidirectional contextual dependencies, preserving local emotional variations, and integrating complementary cross-modal cues. To address these issues, we propose BiM-MFGCN, a bidirectional Mamba and multi-frequency graph convolutional network for multimodal ERC. The model contains a bidirectional Mamba temporal modeling module (BAM) and a multi-frequency graph attention module (FGA). BAM employs forward and backward Mamba2 paths to encode preceding and subsequent dialogue context. FGA constructs an adaptive composite utterance graph, separates low-, mid-, and high-frequency graph-spectral components, and performs cross-modal interaction on frequency-enhanced representations. Experiments on MELD and IEMOCAP show that BiM-MFGCN achieves 66.66% accuracy and 67.82% weighted F1 on MELD, and 70.11% accuracy and 70.12% weighted F1 on IEMOCAP, demonstrating its effectiveness for multimodal ERC.

Read PDF

Similar papers

Book Open access Nov 2026

Conflict-Aware Dual-Path Context Propagation for Multimodal Emotion Recognition in Conversation

Multimodal Emotion Recognition in Conversation (MERC) aims to identify speakers' emotions by integrating conversational context with textual, acoustic, and visual cues. Existing methods remain limited in capturing emotion-relevant contextual dependencies and modeling cross-modal discrepancies during multimodal fusion....

Yang-Tian She, Yu-Ming Lin, Ling-Zhen Wu et al. · 0 citations
Open access Sep 2026

Multimodal transformer and directed relational graph learning for emotion recognition in conversations with potential applications in college student mental health research

Multimodal emotion recognition in conversations aims to identify the emotion expressed in each utterance by jointly analyzing textual, acoustic, visual, and contextual information. However, existing methods still face challenges in coordinating intra-modal dependency modeling, cross-modal interaction, and graph-based c...

Jing Zhao, Feng Liu · 1 citation
Book Open access Nov 2026

HASF-Net: Hierarchical Attention and Spectral Filtering for Multimodal Emotion Recognition in Conversation

Multimodal emotion recognition in conversations (MERC) aims to identify utterance-level emotions from textual, acoustic, and visual cues, and provides a basis for future extension to richer multisensory affective understanding. However, semantic inconsistency across heterogeneous modalities, the under-utilization of mo...

Luntian Mou, Yue Zeng, Sha-Sha Mao et al. · 0 citations
Open access Oct 2026

Application of BERT-EmoShiftNet fusion framework in psychological stress dialogue

Psychological stress dialogue recognition still lacks sufficient semantic understanding in multimodal information fusion and emotional dynamic modelling. It is difficult to achieve high-precision and robust stress state recognition in complex dialogue scenarios. Combining the advantages of the pre-trained language mode...

Cai-Juan Jiao, Huan-Yu Zhao · 0 citations
Oct 2026

Discourse-guided multi-view adaptive fusion for multimodal emotion recognition in conversation

The Discourse-Guided Multi-View Adaptive Fusion framework constructs multiple discourse-guided views and constrains information propagation within each view through discourse-guided masking, thereby reducing irrelevant contextual interference and better preserving fine-grained local emotional cues.

You Li, Yang-Tian She, Li-Hua He et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.