Jul 2026· Proceedings of the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.2· 0 citations· 62 references
Computer Science
TL;DR
Through applied case studies in pharmaceutical discovery and financial systems, common design patterns that make agentic systems successful are analyzed, and practical mitigation strategies for failure modes are discussed, such as verification pipelines, fallback mechanisms, and human-in-the-loop supervision.
Abstract
Agentic systems — large language model (LLM)-based architectures capable of reasoning, planning, acting, and coordinating with tools and other agents — are rapidly transitioning from research prototypes to production-scale deployments across domains such as software engineering, scientific discovery, and finance. While academic work has emphasized benchmarks and algorithmic innovation, deployment raises new challenges around robustness, safety, and reliability. This tutorial brings together researchers and practitioners to explore advances in reasoning and planning, multi-agent coordination, and evaluation, highlighting open challenges arising from deployment experience. Through applied case studies in pharmaceutical discovery and financial systems, we analyze common design patterns that make agentic systems successful, and discuss practical mitigation strategies for failure modes, such as verification pipelines, fallback mechanisms, and human-in-the-loop supervision. Attendees will gain a comprehensive view of the field along with concrete design patterns, evaluation checklists, and templates for safe and reliable deployment across industries.
Large Language Model (LLM)–based agents are rapidly evolving from passive assistants into autonomous, tool-using, and collaborative systems capable of executing complex, long-horizon tasks across web, software, and physical environments. However, the current literature remains fragmented, with inconsistent terminology, ad hoc architectures, and limited evaluation standards, making it difficult to compare systems or deploy them reliably in real-world settings. This paper presents a unified, taxonomy-driven, and deployment-oriented survey of agentic AI systems, synthesizing recent advances through a modular reference architecture and a four-dimensional taxonomy that characterizes agents along the axes of autonomy, tool use, collaboration, and safety–governance. We systematically analyze representative single-agent, tool-augmented, and multi-agent frameworks within this taxonomy, highlighting design trade-offs, capability scaling patterns, and recurring failure modes. Beyond architectural analysis, we review emerging evaluation methodologies that move beyond static benchmarks to assess agent behavior, robustness, grounding, and operational cost in interactive environments. Importantly, the survey emphasizes practical considerations for enterprise and safety-critical deployment, including access control, human-in-the-loop oversight, and policy enforcement. By unifying conceptual foundations with empirical trends and deployment constraints, this work provides a structured roadmap for researchers and practitioners to design, evaluate, and govern next-generation LLM-based agentic systems.
Sparsh Bajoria, Shreyanshu Ranjan, Adhitya M et al.· Cognitive Computation· 0 citations
Agent Gym is introduced, a modular, domain-agnostic framework that wraps any existing LLM-based agent in a continuous evaluation-and-evolution loop and introduces the Spec-to-Note Gap, an autoencoder-inspired view of agentic system transparency.
Pouya Ghiasnezhad Omran, Michael Zimmermann, Duncan Cambridge et al.· 0 citations
It is argued that risk-free deployment must be grounded in the agent's trajectory: the recorded sequence of reasoning steps, tool invocations, and environmental observations, and the absence of adequacy metrics.
Large language model (LLM) agents are increasingly evaluated on their ability to use tools, plan multi-step tasks, coordinate with other agents, and operate over extended horizons. Reported benchmark gains often obscure recurring failure modes documented across otherwise unrelated evaluation efforts. This paper synthesizes 27 benchmark, taxonomy, and audit papers (2023-2026), spanning 19 distinct benchmarks, into a cross-cutting taxonomy of agent limitations. To our knowledge, this is the first synthesis that integrates evidence across tool use, planning, long-horizon reasoning, multi-agent coordination, safety, and measurement validity into a single, unified taxonomy of LLM agent limitations. We identify six failure clusters: (1) tool invocation and parameter-level errors, (2) planning and constraint-satisfaction failures, (3) long-horizon degradation from context accumulation, (4) multi-agent coordination failures, (5) safety and security failures under adversarial or underspecified conditions, and (6) measurement validity problems. The taxonomy was derived iteratively by grouping independently reported error categories into themes corresponding to distinct stages of the agent reasoning-to-action pipeline. Across the literature, we find that failures compound nonlinearly with task length, that strong performance on individual sub-tasks does not reliably translate into end-to-end success, and that additional scaffolding does not consistently improve reliability. At the same time, substantial progress has been demonstrated in single-turn tool use, short-horizon web navigation, and narrowly scoped coding tasks.
Wael S. Albayaydh, Rui Zhao, Ivan Flechais· 0 citations
AgentGym2 is presented, a new evaluation framework with task instances grounded in real-world end-to-end working demands that measures agents'ability to execute end-to-end procedures, discover tools via exploration, compose tools for unseen tasks, and remain robust to noisy and underspecified information.
Zhiheng Xi, Dingwen Yang, Jiaqi Liu et al.· Annual Meeting of the Associ...· 1 citation
Large language models (LLMs) have evolved from standalone generative systems into agentic AI systems capable of planning, reasoning, tool use, and multi-agent collaboration. Enterprises are increasingly adopting AI agents to automate and orchestrate complex workflows, from IT operations to employee productivity. While early deployments focused on proof-of-concept prototypes, the past year has marked a clear shift toward production-grade enterprise AI agents. This transition has been enabled by a wave of new technologies, including multi-agent orchestration, memory and state management, skill-based and modular agent architectures, and deeper integration with enterprise data and workflow platforms, which together make scalable, reliable agent systems feasible in practice. At the same time, moving agents into production introduces new technical and organizational challenges, such as rigorous evaluation and benchmarking, security and governance, and system design for long-running, autonomous operation. Building on the success of our two prior highly attended editions: ''Agentic AI for Enterprise'' workshop at KDD 2025 and ''Enterprise RAG'' workshop at CIKM 2024, this workshop aims to bring together researchers and practitioners to examine how enterprise AI agents can successfully move from prototypes to production. We focus on three pillars: 1) Agent architectures and systems; 2) Enterprise applications and deployments; 3) Evaluation and governance.
Min Du, Anbang Xu, Jasmine Jaksic et al.· Proceedings of the 32nd ACM...· 0 citations