Recent advances in multimodal embodied agents have enabled long-horizon planning in visually rich environments via natural language. Yet, their generalization remains brittle when task instructions deviate from familiar examples, exposing a reliance on surface imitation rather than structural understanding. We propose Causal Abstraction Learning for Multi-Modal Grounded Planning (CALM), a framework that enhances planning agents with the ability to discover and exploit causal regularities across tasks. CALM incrementally develops a causal library by abstracting precondition–effect structure from successful executions, yielding compact representations that emphasize stable dependencies beyond incidental context. When execution diverges from expectation, these abstractions are refined through contrastive causal reasoning, enabling targeted adjustments that resolve underlying mechanism mismatch. The resulting structure serves as a transferable prior for planning in novel settings, integrating perceptual cues with mechanism-informed knowledge. Without retraining or task-specific heuristics, CALM generalizes robustly and efficiently to linguistic and perceptual variation. Experiments on ALFRED and VirtualHome demonstrate consistent gains, highlighting causal abstraction as a scalable inductive bias for grounded planning.
Xinshu Li, Shiyi Yang, Ziqi Xu et al.· Proceedings of the 32nd ACM...· 0 citations
LLM-generated search queries are widely used to augment IR evaluation, yet they may contain concepts that presuppose answer-side document knowledge, violating the information-access boundary of pre-search users. Existing validation metrics, including overlap, diversity, and effectiveness, cannot distinguish rare human-tail variation from candidate answer-side intrusion. We introduce concept provenance, a framework that assigns query concepts to backstory-supported, human-central, human-tail, and candidate answer-side zones, operationalizing a boundary that retrieval metrics alone cannot detect. Applying concept provenance to 77,004 queries across 100 UQV100 topics, 8 LLMs, and 5 prompt conditions with two extraction pipelines, we obtain a cross-pipeline token-HCIR Spearman rho of 1.0 over five condition means. Candidate answer-side concepts constitute 7.40 percent of non-generic concepts and appear in 97 of 100 topics, with topic explaining approximately 67 percent of variance. Human validation yields 68.2 percent relaxed precision, revealing two mechanisms: knowledge intrusion at 45.5 percent and deployment intrusion at 45.0 percent. Diagnostic probes show disproportionate localized retrieval effects, with deletion effect size d = -0.47 compared with d = -0.34 for random deletion, but these concepts explain less than 2 percent of aggregate evaluation variance. Concept provenance therefore serves as a boundary-compliance diagnostic rather than an evaluation-shift predictor. Under the tested conditions, no prompt condition eliminates intrusion; post-generation concept-provenance selection achieves 99 percent elimination.
Chenglong Ma, Xinye Wanyan, Danula Hettiachchi et al.· 0 citations