Carbon-Aware Benchmarking of Prompting Strategies Across Open-Source LLM Families
Abstract
As large language models (LLMs) become widely deployed in real-world applications, prompting strategies such as zero-shot, few-shot, and chain-of-thought (CoT) are commonly chosen primarily based on task accuracy, with little consideration of their environmental impact. This work presents a carbon-aware benchmark that jointly evaluates accuracy, latency, energy consumption, and carbon emissions across six open-source models from three families, namely Llama-3-(3B/8B), Qwen-(4B/8B), and Gemma-4-(4B/8B). Using the CyberMetric cybersecurity dataset as a controlled evaluation testbed, we quantify how different prompting strategies influence computational cost and environmental impact, thereby revealing the resource footprint of prompt design. The empirical findings show that although CoT use more tokens, it does not always result in proportional accuracy gains. Zero-shot prompting outperformed CoT in terms of accuracy and energy efficiency, achieving the highest accuracy while consuming roughly 14× less energy than CoT. These findings are examined via the viewpoint of Pareto efficiency, reframing prompt selection as a systems-level resource management decision. It provides actionable guidance for energy-aware, sustainable LLM deployment in resource-constrained and environmentally sensitive settings.