Skip to content
Preprint

MAC-Bench: Benchmarking Multi-Agent Collaboration via Interconnected Tasks

Nov 2026 · 0 citations
Computer Science

TL;DR

MAC-Bench is introduced, a comprehensive benchmark for evaluating multi-agent collaboration through interconnected tasks that embeds collaboration requirements directly into task dependencies and shared-resource constraints.

Abstract

Large language model (LLM)-based multi-agent systems (MAS) tackle challenging tasks by dividing work among agents and coordinating their contributions toward shared objectives. Evaluating these systems therefore requires benchmarks that assess not only task-solving performance but also the ability to handle interdependent work. However, existing benchmarks often emphasize task difficulty without substantial collaboration requirements or evaluate only a limited range of predefined collaboration patterns. In this paper, we introduce MAC-Bench, a comprehensive benchmark for evaluating multi-agent collaboration through interconnected tasks. Rather than imposing predefined agent roles or communication protocols, MAC-Bench embeds collaboration requirements directly into task dependencies and shared-resource constraints. Specifically, we mine reusable collaboration structures from selected benchmark tasks at two levels. Local collaboration patterns capture recurring interactions among subtasks, while global task structures characterize how these interactions contribute to complete workflows. Through coarse-to-fine composition, these structures are recombined into task graphs combining different collaboration requirements. Tasks are instantiated in executable environments with automated verification of partial progress and complete task resolution.

View source

Similar papers

#artificial intelligence Preprint Oct 2026

MASBench: Benchmarking LLM-based Multi-Agent Collaboration under Partial Observability

MASBench is introduced, a multi-agent collaboration benchmark designed under partially observable constraints that provides deterministic evaluation metrics, including performance score, communication cost, and cost effectiveness, to characterize both collaboration outcomes and communication overhead.

Qi-Zhi Chu, Ze-Kai Yu, Si-Jie Wen et al. · 1 citation
Sep 2026

AgentWorld: Benchmarking Long-Horizon Collaboration of Multi-agent LLMs

To quantify collaboration effectiveness in addition to conventional binary task success, Causal Collaboration Effectiveness (CCE) is proposed, a graph-based metric that traces causal dependencies between agent actions and measures what fraction of a team's effort actually contributed to the outcome.

Raphael Shu, Yu-Sen Zhang, Y. Cho et al. · 2 citations
#artificial intelligence Preprint Oct 2026

Token-Efficient Multi-Agent Collaboration via System One-Guided Computational Division of Labor

Large language model (LLM)-based multi-agent systems (MAS) have become a promising paradigm for complex information-seeking and reasoning tasks by enabling collaborative problem solving among specialized agents. However, existing MAS frameworks tightly couple task reasoning with coordination operations, including task...

Zi-Han Zhou, Xin-Zhe Hu, Han-Xu Yang et al. · 1 citation
#machine learning Preprint Oct 2026

OpenDGO: Benchmarking Multi-Agent Organizations Across Diverse Application Domains

OpenDGO is presented, a controlled benchmark and analysis framework covering six real-world domains and three collaboration settings: Sequential Dependency, Parallel Decomposition, and Information Integration, showing that there is no ``one-organization-for-all''solution.

Xun-Kai Li, Miao Zhang, Ze-Kai Chen et al. · 0 citations
Preprint Oct 2026

Decentralized Multi-Agent Coordination and Communication in Shared Codebases

Coding agents working in shared repositories may receive separate, independently-motivated requests from human developers, creating a setting that is multi-agent by necessity. When this scenario happens, their individual implementations may work separately but conflict when combined. To study this setting, we introduce...

Rishabh Mediratta, Ling-Yun Xiao, P. Stone et al. · 1 citation

DEALS: Decentralized Expertise-Aware Load Serving for Multi-Agent LLM Systems

Experiments show that DEALS not only improves performance along multiple dimensions (e.g., answer accuracy and task throughput) in both homogeneous and heterogeneous agent pools, but also balances agent expertise and workload in a self-organized manner, enabling effective decentralized coordination.

Jing-Hang Huang, Wen-Bin Wang, Yan-Chuan Yin et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.