Wasted Words, Wasted Watts: Babbling and the Avoidable Energy Cost in LLM Code Generation

Abstract

Large Language Models (LLMs) are now widely used in modern software development, assisting developers in generating, testing and maintaining code. However, even when an LLM produces a correct solution, it often continues to generate tokens that do not contribute to it, such as duplicate implementations, usage examples, test code or explanations. We refer to this behavior as babbling. Since LLM inference is energy-intensive such unnecessary output can lead to substantial and avoidable energy consumption. In this paper, we define babbling as the tokens generated beyond the minimal output required to pass all tests. We study it across eight locally-hosted models and two frontier models, four programming languages and four prompting strategies on the HumanEval and BigCodeBench benchmarks. Our results show that the locally-hosted models generate outputs that are on average 390\% longer than the canonical solution, and that 56.4\% of the tokens in correct outputs are babbling. Restricting babbling through lightweight prompting reduces the number of generated tokens, but it comes at a cost in accuracy. The models introduce more formatting errors and tend to prioritize suppressing babbling over solving the task. Lastly, the substantial share of babbling tokens points to a considerable opportunity for energy savings. Based on our results, achieving energy savings without sacrificing accuracy requires more sophisticated approaches than lightweight prompting.