Skip to content
Review

GeoBenchmark: Probing Large Language Models for Geo-Spatial Knowledge

2026 · International Conference on Language Resources and Evaluation · pp. 5335-5348 · 1 citation · 17 references
Computer Science

TL;DR

GeoBenchmark is presented, a benchmark for evaluating geographic commonsense along three core spatial relations: direction, distance, and topology, which demonstrates strong factual recall of general-purpose knowledge but struggle with grounded geospatial knowledge.

View source

Similar papers

#machine learning Preprint Sep 2026

Network-based Spatial Context Retrieval for Open-weight LLMs: A Faithfulness Benchmark for Grounded Geographic Reasoning

Large language models (LLMs) encode substantial latent geographic knowledge, yet they reason poorly over space and are unreliable when queried from coordinates alone. Useful behaviour emerges only when structured spatial context is supplied in the prompt. This raises a question geographic evaluation has left unexamined...

Joan Perez · 0 citations
#artificial intelligence Preprint Oct 2026

Mind the Spatial Gap: A Benchmark for Geospatial Reasoning and Constrained Spatial Editing

Understanding and manipulating spatial relations is fundamental to AI systems that reason about physical environments and construct grounded world models. However, existing evaluations of multimodal large language models (MLLMs) for spatial reasoning rely predominantly on static question-answering (QA) over small-scale...

Zheng Ren, Yan-Ning Wang, Zi-Heng Huang et al. · 0 citations
#natural language process... Preprint Oct 2026

GeopoliticalQA: Benchmarking Retrieval and Generation over Intelligence Reports

Geopolitical and strategic intelligence reports constitute a rich source of expert knowledge and have increasing value in the current political climate, but their heterogeneity and analytical complexity pose challenges for retrieval-augmented question answering. We introduce a benchmark for grounded question answering...

Veronika Solopova, Ivan Havlytskyi, V. Skorik et al. · 0 citations
#artificial intelligence Preprint Oct 2026

GeoFitBench-LLM: A Benchmark for Assessing Large Language Models'Capabilities in Geospatial Data Requirements Elicitation for Geo-Analytic Tasks

Large language models are becoming central to GeoAI systems, yet their evaluation rarely considers the elicitation of geospatial data requirements. This omission is critical as retrieval oriented geo-analytic tasks cannot be supported reliably without first identifying the necessary datasets and constraints. GeoFitBenc...

Amirhossein Vahdat, Thierry Badard, Luc Lamontagne · 0 citations

GeoOutageBench: Benchmarking Ambiguity-aware, Ontology-grounded Geospatiotemporal KGQA for Multimodal Power Outage and Resilience Analysis

GeoOutageBench provides a design principle and foundation for assessing LLM-KG systems that support real-world infrastructure resilience analysis and provides user-configurable evaluation of three important, highly coherent yet less studied tasks.

Ethan Frakes, Amy Kvien, Rishabh Kundu et al. · 0 citations
Open access Aug 2026

GeoGATE: Geo-Sensor-Guided Adaptive Token and Evidence Reasoning for High-Resolution Remote Sensing Image Understanding

GeoGATE is introduced, a geo-sensor-guided framework that combines typed acquisition conditioning, budget-constrained adaptive token acquisition, metadata-compatible evidence retrieval, and reliability-aware temporal reasoning that associates adaptive slicing most strongly with localization, retrieval with language and...

Jing-Nan Zhang, Feng-Jun Zhang · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.