Logical Misconceptions, Pragmatic Insufficiencies in LLMs and How to Fix Them
TL;DR
It is shown that logical incoherencies follow from an LLM’s computation of its internal representations, in particular from an LLM’s failure to take account of the different roles that different expressions may play in determining content.
Abstract
Despite great performance on many tasks, language models (LMs) still struggle with reasoning, sometimes providing responses that cannot possibly be true because they stem from logical incoherence. Extending on the arguments of Asher and Bhar (2024), we show that logical incoherencies follow from an LLM’s computation of its internal representations, in particular from an LLM’s failure to take account of the different roles that different expressions may play in determining content. Linguistics and logicians have shown the importance of the fact that logical operators provide a structure on which to compute content recursively. We extend this view of logical tokens to structure at the discursive level with an eye to improving pragmatic reasoning as well as deductive reasoning. The key in reasoning is that these structures introduce operations over an LM’s latent representations that constrain how they may evolve . We show how LLMs can leverage those structures.