Architectural Patterns for Scalable Multi-Agent LLM Systems: A Software Engineering Perspective

Abstract

The proliferation of large language model (LLM)-based autonomous agents has created a new class of distributed system: the multi-agent LLM network. While significant research focuses on the intelligence of individual agents, comparatively little work addresses the software architectural concerns that govern how fleets of such agents are composed, orchestrated, and scaled. This paper identifies and formalizes four recurring architectural patterns (Hierarchical Orchestration, Event-Driven Agent Mesh, Capability-Indexed Agent Registry, and Stateless Execution with Persistent Memory Offload), drawing on well-established technologies including Apache Kafka, MongoDB, AWS, Spring Boot, and Docker. All experiments were conducted exclusively on synthetically generated data using a locally hosted reference implementation on a personal development machine. No enterprise, production, or personally identifiable data was involved in this work. We analyze each pattern with respect to throughput, fault tolerance, observability, and latency trade-offs using a fully controlled synthetic workload, ensuring complete reproducibility. Our evaluation demonstrates that no single pattern dominates across all dimensions, and that composition of patterns yields the strongest results. This work provides a foundation for treating multi-agent LLM system design as a first-class software engineering discipline. Note: The four patterns operate at complementary abstraction levels coordination (Patterns 1 and 2), discovery and routing (Pattern 3), and state management (Pattern 4) and are designed for deliberate composition rather than as parallel alternatives.