Language-Grounded Strategy-Following Multi-Agent Deep Reinforcement Learning for Controllability of Real-World Applications

Abstract

We present lg-sfDA6-X, a language-grounded strategy-following distributed attentional actor architecture after conditional attention, for multi-agent deep reinforcement learning (MADRL). The proposed architecture aims to enable controllable and coordinated agent behaviors in application systems by leveraging a shared saliency representation that integrates environmental conditions with high-level textual instructions provided by external experts and users. To this end, we introduce language-based destination strategies, which allow agents to adapt their behaviors simply by specifying the target regions of the environment using natural language. We evaluate lgsfDA6-X in an object collection game and analyze how agents modify their cooperative and coordinated behaviors in response to diverse, previously unseen, and compositional instructions by users. Experimental results demonstrate that lg-sfDA6-X effectively grounds linguistic instructions into semantically structured latent representations, enabling robust strategy following and coordination. These findings suggest that lg-sfDA6-X provides a promising approach for achieving flexible and interpretable control of coordinated behaviors in MADRL through natural language instructions.