CEMAS - a Configurable Embodied Multi-Agent System

Abstract

This work presents CEMAS, a Configurable Embodied Multi-Agent System that enables one-to-many interactions between a human user and multiple Embodied Conversational Agents (ECAs), allowing the simulation of complex social interactions in extended reality (XR). The system comprises two layers: (1) an AI interaction layer that provides a modular, server-side voice-to-voice pipeline for dynamic verbal behavior together with a moderator-based turn-taking mechanism to coordinate multi-party conversations, and (2) an embodiment layer that manages the visual representation of ECAs in Unity and executes AI-generated non-verbal behaviors synchronized with spoken output. By combining modular conversational AI with coordinated multi-agent interaction and embodied behavior generation, CEMAS provides a configurable foundation for developing socially interactive multi-agent systems in XR and facilitates future research and applications in domains such as education, mental health, and collaborative human-AI decision making.