2026· International Conference on Language Resources and Evaluation· pp. 5335-5348· 1 citation· 17 references
Computer Science
TL;DR
GeoBenchmark is presented, a benchmark for evaluating geographic commonsense along three core spatial relations: direction, distance, and topology, which demonstrates strong factual recall of general-purpose knowledge but struggle with grounded geospatial knowledge.
Large language models (LLMs) encode substantial latent geographic knowledge, yet they reason poorly over space and are unreliable when queried from coordinates alone. Useful behaviour emerges only when structured spatial context is supplied in the prompt. This raises a question geographic evaluation has left unexamined...
Understanding and manipulating spatial relations is fundamental to AI systems that reason about physical environments and construct grounded world models. However, existing evaluations of multimodal large language models (MLLMs) for spatial reasoning rely predominantly on static question-answering (QA) over small-scale...
Zheng Ren, Yan-Ning Wang, Zi-Heng Huang et al.· 0 citations
Geopolitical and strategic intelligence reports constitute a rich source of expert knowledge and have increasing value in the current political climate, but their heterogeneity and analytical complexity pose challenges for retrieval-augmented question answering. We introduce a benchmark for grounded question answering...
Veronika Solopova, Ivan Havlytskyi, V. Skorik et al.· 0 citations
Large language models are becoming central to GeoAI systems, yet their evaluation rarely considers the elicitation of geospatial data requirements. This omission is critical as retrieval oriented geo-analytic tasks cannot be supported reliably without first identifying the necessary datasets and constraints. GeoFitBenc...
GeoOutageBench provides a design principle and foundation for assessing LLM-KG systems that support real-world infrastructure resilience analysis and provides user-configurable evaluation of three important, highly coherent yet less studied tasks.
GeoGATE is introduced, a geo-sensor-guided framework that combines typed acquisition conditioning, budget-constrained adaptive token acquisition, metadata-compatible evidence retrieval, and reliability-aware temporal reasoning that associates adaptive slicing most strongly with localization, retrieval with language and...