Energy constraints in large scale artificial intelligence and insights from human cognition for efficient large language models

Abstract

Abstract Large-scale artificial intelligence (AI) systems, especially large language models (LLMs), face growing constraints related to energy use, power delivery, memory traffic, experimentation, and infrastructure capacity. These pressures are often studied separately, which makes it difficult to understand how they interact across the full AI lifecycle. This review develops a cross-layer framework that brings these constraints together and examines whether resource-allocation strategies observed in human cognition can inform more sustainable AI design. We compare human and artificial systems by function rather than anatomy, focusing on how each allocates limited resources to learning, search, memory, external support, optimization, and inference. Using a systematized narrative-review approach, we identified 6284 records, screened 898 at the full-text stage, and selected an initial set of 150 focal studies for detailed synthesis. We organize the evidence into seven interacting regimes covering scaling, experimentation, hardware power limits, data movement, infrastructure, evaluation incentives, and inference allocation. The framework extends Green AI by linking these layers and translating insights from human cognition into testable design hypotheses and measurable evaluation criteria. The strongest near-term opportunities include data selection, energy-aware experimentation, hardware-supported low-precision computation, and demand-adaptive inference. However, most reported gains are based on proxy measures such as operations, memory, latency, or data volume rather than direct end-to-end energy measurements. Near- and in-memory computing show longer-term promise, while retrieval, tool use, distributed experts, and multi-agent systems may shift or increase energy costs through communication and repeated inference. These findings support cross-layer co-design and end-to-end evaluation at matched capability, reliability, latency, and service quality.