A Reproducible Benchmark Protocol for Autonomous Micromobility Local Planning in Shared Pedestrian Spaces

Abstract

Autonomous micromobility vehicles (AMVs) need local planning that balances task progress, safety, and pedestrian interaction quality in pedestrian-rich shared spaces. Evaluating such planners is difficult because studies vary scenarios, seeds, metrics, outputs, and aggregation rules, while single-score leaderboards hide which behaviors produce a ranking. This paper proposes a repeatable, auditable, multi-objective benchmark protocol for AMV local planning. Before comparison, it fixes the scenario set, repeated seeds, measured metrics, stored outputs, planner-interface records, and aggregation procedure. We demonstrate it with a frozen robot_sf_ll7 campaign: 47 shared-space scenarios, three evaluation seeds, and 141 scenario-seed episodes per planner in one differential-drive AMV configuration. The stress test surfaces a descriptive safety–performance separation: a Proximal Policy Optimization (PPO)-family profile reaches higher observed mean task success than a classical reciprocal-avoidance baseline, while that baseline keeps lower collision exposure. Absolute success stays low for both, with most scenarios unsolved by either. Because this learned policy was trained on a superset of the evaluation scenarios, its higher success reflects behavior on the benchmark set, not held-out generalization—an overlap the protocol records per planner rather than hiding in one score. The finding is bounded to these configured pipelines, not a universal planner-family ranking. The contribution is an auditable comparison framework tracing results from the scenario matrix and fixed seeds to episode records, aggregate reports, and manifests.