LLMs are metaphor machines: orbital argumentation, misaligned citations, and scientific fabrication in AI-generated writing

Abstract

Large language models are metaphor machines: they generate text based on similarities and proximity in their training data. I argue that this metaphor-based text generation leads to what John Gallagher calls orbital argumentation , where the generated text is pulled towards key concepts that, I argue, remain unnarratable as the text skims their surface, orbiting around them instead of directly addressing them. Misaligned citations in scholarly writing—citations of real works that are related to the central topic but do not support the specific claim made—are a type of scientific fabrication that is systematically produced by the fundamentally metaphorical operation of large language models. To develop this argument, I draw upon theories of metaphor by Aristotle and I. A. Richards, applying them to two technological shifts that underpin the metaphorical emphasis of LLMs: the 2013 discovery that language models could handle analogies like king − man + woman = queen and the introduction of self-attention and transformer models in 2017. I use this framework to analyse an LLM-generated document summary, examples from a dataset of LLM-generated stories, and a scientific paper. In each case, I demonstrate how the metaphorical connections pull at the text generation, causing the text to orbit around a key theme that remains unnarratable. Recognising the rhetorical vices of LLM-generated texts can help us to protect the knowledge ecosystem that science depends upon. The paper, therefore, concludes with five recommendations for readers, writers, peer reviewers, and editors who want to build resilience against the inaccuracies of LLM-generated texts.