Role-aware Heuristic Episodic Attention for Conversational LLMs
Large language models often lose track of persistent instructions and relevant information as multi-turn conversations grow. We study this cumulative contextual decay through three related failure modes: attention pollution, dilution, and drift. We propose REA (Role-aware Heuristic Episodic Attention), a context-manage...
Wan-Yang Hong, Zhao-Ning Zhang, Yi Chen et al.
· 0 citations