Skip to content
Preprint

ESCRAG-R1: Retrieval-Augmented Reinforcement Learning for Emotional Support Conversation

Aug 2026 · 1 citation · 36 references
Computer Science

TL;DR

ESCRAG-R1 is proposed, a unified framework that integrates retrieval-based psychological guidance into Group Relative Policy Optimization (GRPO) and transforms external knowledge into a robust learning signal that stimulates explicit internal reasoning prior to generation and fundamentally reshapes the model's internal policy.

Abstract

Emotional Support Conversation (ESC) systems aim to provide holistic support by balancing professional therapeutic competence with natural empathy. However, existing methods struggle to simultaneously achieve structured, stage-aware reasoning and seamless empathy-expertise alignment, often resulting in an artificial splicing of clinical strategies and generic reassurance. To overcome these limitations, we propose ESCRAG-R1, a unified framework that integrates retrieval-based psychological guidance into Group Relative Policy Optimization (GRPO). By incorporating retrieval into the reinforcement learning loop, ESCRAG-R1 transforms external knowledge into a robust learning signal that stimulates explicit internal reasoning prior to generation and fundamentally reshapes the model's internal policy. To provide the reliable supervision required for this optimization, we construct ESC-Preference, a high-quality dataset based on a Client--Counselor--Judge evaluation framework that delivers precise, empathy-aware reward signals. Extensive experiments demonstrate that ESCRAG-R1 significantly outperforms existing baselines by mitigating superficial splicing and realizing a natural integration of professional guidance and empathetic expression. Code and datasets are released at https://github.com/Matcha-Liu/ESCRAG-R1.

View source

Similar papers

Sep 2026

DSPO: Diversity-aware Subjective Policy Optimization for Robust Emotional Reasoning

DSPO, a reinforcement learning framework that jointly promotes subjective affective coverage and visual grounding, achieves state-of-the-art performance across multiple public benchmarks, especially on the cross-domain performance, i.e., improving +10.8\% on average cross-domain accuracy than EMO-R3.

C. Ye, Wei-Dong Chen, Bing-Yan Xu et al. · 0 citations
Review Aug 2026

DeepSAGE: Stage-Aware Reinforcement Learning for Structured CBT Counseling Dialogue

DeepSAGE (Strategic AI Guidance Engine), a hybrid LLM--Deep Reinforcement Learning (DRL) framework for stage-aware counseling dialogue grounded in the first session of Cognitive Behavioral Therapy (CBT), suggests that combining stage-structured dialogue with learned strategy selection is a promising approach for AI cou...

Qi Zhang, Heajun An, P. Dumaru et al. · 0 citations
Open access Oct 2026

Emotionally intelligent preference optimization for conversational behavior change support

Procrastination negatively affects productivity and well-being, yet current large language models (LLMs) often lack the nuanced emotional understanding necessary for effective behavior-change support. This study proposes emotionally intelligent preference optimization (EIPO), a unified framework integrating emotion...

Ben Chen, Nina Dethlefs · 0 citations
Open access Sep 2026

Beyond Labels: Training Cognitive Empathy with Context-Rich Synthetic Dialogues

RelationalDialogues is introduced, a novel, fully synthetic dataset of 12,849 multi-turn dialogues designed to explicitly train perspective-taking, demonstrating that training on highly contextualized, metadata-driven synthetic data is an effective method for advancing LLMs from displaying superficial sympathy to engag...

Neema Owji, Cameron Buckner · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.