Does the Evidence a Model Cites Drive Its Conclusion? A Pre-registered Causal Test of Declared Evidence Relations in Structured Reasoning Traces

Abstract

When a language model writes down its reasoning, does the evidence it cites actually drive the conclusion it states? We contribute a reproducible method for testing causal alignment between declared trace relations and model computation, and a pre-registered application to Qwen2.5-7B-Instruct on a controlled comparison task. Models emit typed, machine-checkable reasoning traces in Scholialang, where a Finding atom declares its dependence on an Evidence atom. We test that declared edge by activation patching: we splice a counterfactual Evidence span's activations into the clean context and check whether the Finding's read is redirected as the relation predicts. For this model and task, a model's emitted symbolic label can causally steer its subsequent conclusion, even when the accompanying facts disagree. On the same 90 self-generated chains, patching the cited span redirected the conclusion in 77/90 relations when the twin flipped both numbers and label. Flipping only the polarity label (supports/refutes) redirected it in 73/90 (paired with the full twin: 84.4% agreement, McNemar p = 0.424). Swapping only the numbers never changed the selected status under this intervention (0/90), although in a post-hoc analysis it shifted the decision margin toward the counterfactual status in every case, by about 27% as much as the label flip. A donor that changed only the Evidence text redirected exactly the same 77 relations as the full twin. What we establish is downstream label influence. Where the numeric comparison is computed, and whether the declared reference structure itself controls routing, remain open: we did not intervene on how the label was generated. The supporting results were pre-registered. Validation, where task relevance is known by construction, separates relevant from irrelevant evidence (J = 1.000 in both evidence orders; per-class Wilson lower bound ≈ 0.862). Redirection rates on two batteries were 77/90 and 53/71 (roughly 75–86%, with substantial between-battery variance). Redirection also succeeds when the relevant evidence is second, although broader position effects are not ruled out. Every redirection failure showed a joint evidence-plus-observation response, and unrelated-evidence controls never redirected. Behavioral arms on two hosted models show direction-sensitivity to their own evidence edits, but an authority-cue arm induced no unfaithfulness, so we have not shown that the method detects genuine unfaithfulness.