Skip to content
#robotics #human-computer interaction Preprint Open access

FleetScape: A Mixed Reality Sandtable for Spatial Supervision and Control of Scalable Drone Fleets

Peisen Xu J\'er\'emie Garcia Peter Cleveland Wei Tsang Ooi Christophe Jouffrais
Aug 2026
Robotics Human-computer Interaction

Abstract

As autonomous drone deployments scale from individual units to coordinated swarms, the human operator's role shifts from direct piloting to high-level supervision. Current interfaces often treat multi-drone control as a scaled-up version of single-drone operation. We instead investigate how reframing fleet supervision as spatial interaction can better support the spatial, temporal, and safety demands of complex missions. We present FleetScape, a Mixed Reality (MR) sandtable system that externalizes layered real-time mission, safety, and environmental data while enabling fluid transitions between manual intervention and autonomous supervision. We developed a high-fidelity building inspection simulation that generates and streams synchronized multi-drone and environmental data for MR visualizations. We used this prototype to conduct a user study with six experienced drone pilots managing fleets of up to 15 drones. Our findings show that FleetScape supports situational awareness through layered spatial representations and clarifies control mode transitions. However, a limit to situational awareness was observed as fleet size increases, leading to different supervisory strategies. Finally, we derive design implications for supporting scalable drone fleet supervision.

View source

Similar papers

#artificial intelligence Review Dec 2025

Professional Software Developers Don't Vibe, They Control: AI Agent Use for Coding in 2025

Investigating how experienced developers use agents in building software, including their motivations, strategies, task suitability, and sentiments finds that while experienced developers value agents as a productivity boost, they retain their agency in software design and implementation out of insistence on fundamental software quality attributes.

Ruanqianqian Huang, Avery Reyna, Sorin Lerner et al. · 19 citations · ⚡1
#artificial intelligence Preprint Jul 2026

Mask2Real-WM: Segmentation Masks as a Sim-to-Real Bridge for Controllable Dexterous World Models

Mask2Real-WM is presented, a two-stage action-conditioned world model for dexterous manipulation that decouples pixel prediction into a dynamics model and a rendering model that shows that mask conditioning and simulation pretraining are both required for per-DoF action controllability across all 23 degrees of freedom.

Riccardo Feingold, Davide Liconti, Chenyu Yang et al. · 1 citation

Planning-aligned Token Compression for Long-Context Autonomous Driving

This work proposes COMPACT-VA, a planning-aligned working memory framework built on conditional VQ-VAE, compressing extended context into bounded representations, and evaluates on high-signal dynamic scenarios where historical context is most critical for behavior correctness, and accordingly design behavioral metrics.

Zhixuan Liang, Yuxiao Chen, Yurong You et al. · 1 citation
#artificial intelligence Preprint Open access Aug 2026

Efficient Dynamic Shielding for Parametric Safety Specifications

Shielding has emerged as a promising approach for ensuring safety of AI-controlled autonomous systems. The algorithmic goal is to compute a shield, which is a runtime safety enforcement tool that needs to monitor and intervene the AI controller's actions if safety could be compromised otherwise. Traditional shields are designed statically for a specific safety requirement. Therefore, if the safety requirement changes at runtime due to changing operating conditions, the shield needs to be recomputed from scratch, causing delays that could be fatal. We introduce dynamic shields for parametric safety specifications, which are succinctly represented sets of all possible safety specifications that may be encountered at runtime. Our dynamic shields are statically designed for a given safety parameter set, and are able to dynamically adapt as the true safety specification (permissible by the parameters) is revealed at runtime. The main algorithmic novelty lies in the dynamic adaptation procedure, which is a simple and fast algorithm that utilizes known features of standard safety shields, like maximal permissiveness. We report experimental results for a robot navigation problem in unknown territories, where the safety specification evolves as new obstacles are discovered at runtime. In our experiments, the dynamic shields took a few minutes for their offline design, and took between a fraction of a second and a few seconds for online adaptation at each step, whereas the brute-force online recomputation approach was up to 5 times slower.

Davide Corsi, Kaushik Mallik, Andoni Rodriguez et al. · 0 citations
#machine learning Preprint Aug 2026

Repetition as Reinforcement: Enhancing Sample Efficiency via Instant Episode Repetition in Reinforcement Learning

Instant Episode Repetition (IER) is introduced, a simple and novel mechanism that improves sample efficiency by immediately repeating action sequences from successful episodes during environment interaction by directly influences the data collection process.

Hoda Yamani, Yuning Xing, Koen van Rijnsoever et al. · 0 citations
#computer vision Preprint Open access Aug 2026

A Simulator-Grounded Framework For Constructing Verifiable Muscle-Grounded QA From 3D Tongue Meshes (extended version)

Existing articulatory corpora based on real-time MRI and electromagnetic articulography capture tongue motion but lack traceable labels for the muscle-driven process underlying each configuration. We introduce a simulator-grounded construction framework, instantiated as 3DTongueQA. Controlled 11-D muscle activations are mapped by the ArtiSynth Badin finite-element model to fixed-topology tongue meshes, converted into reusable biomechanical records, and rendered as deterministic QA on simulator-defined muscle state, geometry, and target-directed change. From 295,157 screened configurations, 295,115 valid meshes yield 891,156 QA records per language. English and Korean naturalizations modify only surface form and are verified against the source records. A swappable SpiralNet++--Qwen3-8B baseline reaches $62.9\pm9.2$ Muscle EM, dropping to 2.2 under mesh shuffling; anchor-held-out training retains 80.4--98.6\% of full-inventory scores, while task-specific readouts reach $88.7\pm0.7$ Muscle EM and $93.3\pm1.0$ Direction EM. The results show geometry-grounded supervision supporting both unified language QA and structured prediction. Code is available at https://github.com/esh0504/muscle-grounded-qa .

Seungho Eum, Unsang Park · 0 citations

Related blog posts