Jul 2026· SIGGRAPH Posters· pp. 1-3· 0 citations· 11 references
Computer Science
TL;DR
GAIN-AI (Guided Assistant for Intelligent Navigation), a context-aware AI assistant and minimal heads-up interface for procedural guidance in simulated lunar EVA, is presented.
Abstract
As human space exploration returns to the Moon, astronauts need rapid access to procedural information during extravehicular activities (EVAs), where attention is divided across navigation, repair tasks, tool handling, and environmental risk. The challenge is not the absence of information, but surfacing the right information at the right moment. We present GAIN-AI (Guided Assistant for Intelligent Navigation), a context-aware AI assistant and minimal heads-up interface for procedural guidance in simulated lunar EVA. The system operates in two layers. The first grounds a large language model with structured context: EVA procedure documents, live telemetry data, and error-handling protocols encoded as JSON. The second restructures that output into three compact units for AR display: Goal, Task, and Verification. Evaluated on 111 synthetic EVA scenarios, the system scores 10.0/10 on nominal conditions and 8.15/10 on single-fault scenarios, with performance degrading on multi-fault and boundary-threshold cases.
Future human exploration of Mars faces a critical operational barrier: communication latencies of 3 to 22 minutes will sever the real-time feedback loop between the crew and Earth-based Mission Control Center (MCC), rendering traditional “over-the-shoulder” guidance impossible during Extravehicular Activities (EVA). Building on our prior RASAGE (Retrieval & Simulation Augmented Guidance Agent for Exploration) work, we propose an onboard, multi-agent architecture that replicates key MCC console functions and their information pathways while operating under edge power, memory, and latency limits. This system decomposes the monolithic MCC structure into specialized agents, including the Flight Director, CAPCOM, and Systems Specialists, integrated with a Hybrid Retrieval-Augmented Generation with dedicated knowledge graph framework. To mitigate hallucination risks, the architecture employs deterministic tool use grounded in the NASA Crew State & Risk Model (CSRM), ensuring physiological validity and traceability to flight rules. System reasoning and orchestration are executed with Gemini 2.0 Flash to enable low-latency, long-context inference suitable for edge deployment, while evaluation uses Claude Opus 4.5 as an evidence-checking judge for long-horizon verification against source documentation. We validated the system by reconstructing the Apollo 14 EVA missions with a knowledge graph derived from over 50 historical mission documents. Across a benchmark of 244 operational queries, the system achieved an overall pass rate of 78.7%, with 94.6% accuracy on procedural inquiries and a hallucination rate below 0.4%. These results demonstrate the feasibility of transforming centralized ground support into a resilient, autonomous partner capable of safeguarding crew during high-latency planetary exploration.
Kaisheng Li, R. Whittle· 55th International Conferenc...· 0 citations
Future human missions to Mars will place astronauts in a world that is scientifically rich but
physically unforgiving. The Martian surface has a thin atmosphere, extreme temperature swings, dust
activity, radiation exposure, delayed communication with Earth, limited resupply, and full dependence
on engineered life-support systems. These conditions make Mars exploration a safety-critical, distributed
teamwork problem rather than a simple task-planning problem. This article develops a conceptual
framework for human-artificial intelligence (AI) teamwork in Mars surface exploration. No detailed
mathematical model is proposed, no AI system is trained, no operational performance is claimed, and
no crewed test has been performed. Instead, the contribution is a research-grounded engineering concept
organized around Figure 1, in which mission readiness, in-mission support, human review, spacecraft
and habitat architecture, and data-to-value feedback form a closed safety loop. The framework argues
that AI should not replace astronauts or mission-control teams; rather, AI should act as a safety translator
that turns environmental data, system telemetry, robot reports, and science priorities into explainable
options for human approval. The proposed concept adds engineering soundness by mapping Mars
hazards to decision-support functions, defining human-authority requirements, identifying safety
and planetary-protection guardrails, and laying out a staged validation pathway from concept review
to tabletop exercises, analog missions, digital twins, and eventually certified operational systems.
The central message is that the smartest path to Mars is neither full automation nor unaided human
courage, but disciplined human-AI-robot teamwork that helps explorers remain safe, aware, ethical, and
scientifically productive.
Junyao Zhang, Hongsheng Shang· American Journal of Student...· 0 citations
Multi-drone systems are increasingly positioned for safety-critical missions such as search and rescue (SAR) and critical infrastructure monitoring. Yet, real-world adoption remains constrained not only by autonomy performance, but by the difficulty of integrating agentic behavior into professional work: operators must understand, trust, and govern automation under uncertainty, time pressure, and accountability. This position paper synthesizes the ambitions and lessons from two ongoing efforts: NAMUR, which explores LLM-supported robot control in SAR and firefighting contexts, and PERSIST, which explores persistent drone operations for monitoring and security at critical infrastructure sites. We argue that agentic AI should be approached as a socio-technical design problem, where interfaces, oversight mechanisms, and evaluation practices are as critical as algorithms. We outline a human-centered, participatory, and iterative research approach aimed at uncovering stakeholder needs, shaping agent capabilities through successive prototypes, and producing transferable proof-of-concept systems and evaluation strategies for other safety-critical contexts.
T. Merritt, Alejandro Jarabo-Peñas, Juan Bravo-Arrabal et al.· 0 citations
A safety-gated evaluation framework in which a trajectory succeeds only when all task goals are achieved without violating any hard safety constraint, while safe goal progress and trajectory safety are measured separately is established.
Yuchen Yuan, Zhenghuang Wu, Yuangan Li et al.· 0 citations
High-tempo domains such as defense, medicine, and transportation have been transformed by AI-based systems that accelerate information processing and decision support. Modern air combat reflects this evolution, with automated systems continuously generating tactical options that require aircrews to decide more frequently under time pressure. This shift is exemplified by the deployment of Autonomous Collaborative Platforms (ACPs), i.e., AI-enabled teamed drones operating alongside crewed fighters and proposing actions such as target validation. This study examines whether the processing of target-validation situations proposed by ACPs can be improved through cognitive training. Forty military aircrew members participated in a virtual reality flight mission and were assigned to either a control or a training group. Both groups completed identical pretest and posttest separated by 24 hours, while only the training group performed a 45-minute computer-based training session focused on repeated target-validation tasks. Results show that training significantly reduced error rates and response times, in line with theories of expertise acquisition. In contrast, perceived workload decreased similarly in both groups, suggesting a dissociation between objective performance gains and subjective workload. These findings support the relevance of cognitive training for preparing aircrews for future collaborative combat involving ACPs.
Jean-christophe Hurault, Marianne Jarry, G. Froger et al.· AHFE International· 0 citations
We propose HODAgent, a System-2 embodied agent for humanoid robots in service settings, addressing situated intent, responsive execution, task revision, and outcome verification. Its semi-duplex architecture integrates an Env-Interactor, Planner, Executor, and hierarchical Memory to maintain coherent interaction, planning, and task state during service episodes. This allows handling new requests during motion, retaining progress, revising actions, and grounding closure in execution outcomes. A shared interface connects simulation and physical robots (Unitree G1), isolating platform-specific control. In an interactive simulation with 164 cases, HODAgent achieves 84.8% and 91.5% Joint Success under two VLM backbones, outperforming baselines by 9.8 and 18.9 points. On physical robots, pass rates are 92% (atomic), 72% (composite), and 63.3% (complete tasks). On multiple embodied benchmarks, it improves over baselines by 0.7-9.0 points. Results show a unified System-2 agent enables adaptive humanoid service across simulation and reality.
W. Chen, Jiahao Zhang, Zhenjiang Li et al.· 0 citations