An Empirical Evaluation of LLM Function Calling in Enterprise Backend Systems: Latency, Reliability, and Development Trade-offs

Abstract

Function calling enables Large Language Models (LLMs) to orchestrate backend services by invoking structured tools, extending their role from text generators to intelligent agents. While prior work evaluates tool-use accuracy, there is limited empirical evidence on the backend-level performance trade-offs between LLM-driven orchestration and traditional deterministic implementations in enterprise systems. This study presents a controlled, reproducible benchmark comparing Javabased deterministic backend logic with LLM function-calling pipelines using locally deployed open-weight models (Mistral 7B, Llama 3.1 8B, and Qwen 2.5 7B). Across three representative enterprise scenarios, data retrieval, data normalization, and command execution, we measure accuracy, latency, error rate, token efficiency, and development effort. Results show that deterministic implementations achieve near-instantaneous execution (0.2-1.8 ms) with 100% reliability, while LLM-driven approaches incur a $\mathbf{1 0 0} \times \mathbf{- 1 6 0 0} \times$ latency overhead (200-800 ms per request). Because of this significant delay overhead, LLM function calling is inappropriate for high-throughput and latencycritical backend services. However, selected 8B-class models (Llama 3.1 and Qwen 2.5) maintain 100% schema adherence under zero-temperature settings and reduce codebase size by approximately 33%, improving development agility. These findings demonstrate that LLM function calling is not a replacement for deterministic backend logic in latency-critical systems, but rather an effective orchestration layer for dynamic and natural language-driven workflows. We conclude by proposing a hybrid backend architecture that combines compiled logic for highperformance transactional paths with LLM-based reasoning for flexible interaction layers.