Pragmatic AI: building low-impact LLM tools for energy research impact

Abstract

Extended abstract 3-063-26 Large language models (LLMs) provide a powerful means to search for and summarise content. However, their training and use is a rapidly growing driver of energy and resource use. In this presentation we describe the development of an LLM-based search tool for the UKRI Energy Demand Research Centre (EDRC), with a view to demonstrating how the important benefits such tools enable (such as supporting impact activities) can be reasonably balanced with resource implications.  The tool is based on retrieval augmented generation (RAG). All EDRC outputs, along with those of its predecessor the Centre for Research into Energy Demand Solutions, were broken into shorter sections (“chunks”), which in turn were encoded as vector embeddings. These capture the semantic meaning of each chunk in a quantitative fashion. When a user searches using the tool, the chunks most closely related to their query are first identified – and only then are they submitted to an LLM to check for relevance and to create a summary. This allows a large corpus of text to be processed with minimal energy use, cost, and less of the hallucinations which become more likely as larger quantities of text are processed. Smaller, low-energy-intensity LLMs are employed where possible, and users are provided with feedback on the estimated energy and resource impacts of their searches.  Potential uses of the tool include internal and public search and information retrieval, as well as more advanced applications such as constructing preparatory briefings for consultation responses. We are expecting to introduce further features before the Summer Study. While the tools we used are widely adopted, we believe they are still under-employed in the energy demand community, in part due to concerns around their resource impacts. As such, as well as introducing the approach to those unfamiliar with it, we also hope to foster debate about when and how LLMs can be appropriately employed. Download presentation.