TOPO-Bench: An Open-Source Topological Mapping Evaluation Framework with Quantifiable Perceptual Aliasing
TL;DR
This work formalizes topological consistency as the fundamental property of topological maps and shows that, under mild assumptions, localization accuracy provides an efficient and interpretable surrogate metric, and introduces the first quantitative measure of dataset ambiguity for fair comparison across environments.
Abstract
Topological mapping offers a compact and robust representation for navigation, but progress in the field is hindered by the lack of standardized evaluation metrics, datasets, and protocols. Existing systems are evaluated in different environments under different criteria, preventing fair and reproducible comparison. Moreover, a key challenge—perceptual aliasing—remains under-quantified despite its strong influence on system performance. We address these gaps by (i) formalizing topological consistency as the fundamental property of topological maps and showing that, under mild assumptions, localization accuracy provides an efficient and interpretable surrogate metric, and (ii) introducing the first quantitative measure of dataset ambiguity for fair comparison across environments. To support this protocol, we curate a diverse benchmark dataset with calibrated ambiguity levels, implement and release deep learning-based baseline systems, and evaluate them alongside classical methods. Our experiments provide new insights into the limitations of current approaches under perceptual aliasing. All datasets, baselines, and evaluation tools are publicly released to foster consistent and reproducible research in topological mapping.
