Retrieval Augmented Generation for Relative Distance-based Spatial Reasoning

Abstract

Large Language Models (LLMs) can be applied to various real-world tasks, many of which require spatial reasoning, like pattern-based spatial search, Point of Interest (POI) recommendation, and itinerary or travel planning. However, on their own LLMs have been shown to lack reliable spatial reasoning capabilities, especially about distances. Further, success at these tasks requires the LLM to have been exposed to the relevant places and to successfully disambiguate their spatial interpretations, which is affected by the popularity of the place name. To address these problems, we develop a novel approach using spatial Retrieval Augmented Generation (RAG) to enable an LLM to use relevant distance information not explicitly learned during training when answering spatial questions that involve distance reasoning. Our method encodes a spatial graph of geodesic and road network distances between cities and towns, as well as a mapping to their alternative names, and compares a variety of techniques to provide the LLM with the appropriate spatial context from those knowledge stores. By incorporating relevant spatial information, our method significantly outperforms the baseline LLM on several tasks requiring distance-based spatial reasoning, including nearest-neighbor questions and distance analogy questions. We further show through experiments which graph storage and retrieval methods perform best, and which are brittle when applied to more challenging questions. Given the vast array of possible places an LLM could be asked to reason about, our work offers a first step towards providing a rudimentary distance-oriented spatial ‘world model’ to complement the vast linguistic knowledge held in LLMs.