Skip to content
Preprint

Causal Structure is Inducible but Functionally Decoupled: The Routing/Readout Boundary of a Typed Mechanism Library

Aug 2026 · 0 citations · 28 references
Computer Science

TL;DR

A decoupling in how a transformer organizes causal knowledge is reported: slot-by-type structure induced by type-level supervision organizes routing, yet remains functionally decoupled from answer readout.

Abstract

When a language model answers an interventional question, the computation it must perform depends on the type of evidence the query requires. We report a decoupling in how a transformer organizes causal knowledge: slot-by-type structure induced by type-level supervision organizes routing, yet remains functionally decoupled from answer readout. We establish this with a typed mechanism library -- discrete mechanism slots partitioned by evidence type, auditable at the state level -- on a causal-world benchmark with exact interventional ground truth, under a frozen protocol, at two scales (22.6M and 125M). Four preregistered findings. (i) Origin. Slot-by-type organization is induced by type-level supervision: absent in architecturally identical unsupervised controls, not buyable by content-free gating labels, and statistically attributable to the supervision signal, replicating at 125M under a powered preregistered protocol (all nine cells passed). (ii) Boundary. The induced structure is a typed routing index with a sharp routing/readout boundary: slot codes scaffold routing but do not drive answer readout ($|\Delta\hat{y}| \le 3.4\times10^{-6}$, zero collateral, three seeds, stable across a 5.6x scale window) -- we therefore make no behavioral-editability claim. (iii) Cost. The structure is free: LM quality matches a parameter-matched monolith within 0.0082 nats. (iv) Trust. The library state is exactly local under edit and bit-exactly revertible -- 250 single-edit and 1,000 stacked reverts per seed, zero failures. We further find that the unsupervised null itself moves with scale, so comparisons reusing a null calibrated at one scale may be confounded at another. Every claim is tied to a preregistered, machine-checkable criterion archived before the data it governs; the full audit trail, including one criterion we failed and how the frozen protocol handled it, is released as an appendix.

View source

Similar papers

Preprint Aug 2026

Beyond the Trace: Coupling an Interpretable Reasoning-State Readout to Native MoE Routing

A two-level internal readout for mixture-of-experts reasoning that makes latent process state readable, while routing makes it deployable and actionable and improves single-branch selection and plain majority voting over completed candidate sets.

Kang Chen, Si-Han Zhao, Yi-Xin Cao et al. · 0 citations
Preprint Aug 2026

Located but Not Releasable: Silent Gate Inversion and Bounded Linear Release

The complete pipeline -- detect, localize, and release -- is submitted to a fully preregistered stress test on a 25.7M transformer trained on causal-evidence discrimination, where a known suppression phenomenon (latent causal structure present but behaviorally unused) has previously been documented.

Xi-Ning Xun · 0 citations
Jul 2026

A Compositional Theory of Causally Masked Transformers

This work develops an algebraic formalization that derives expressivity directly from the model's implemented dynamics, and obtains an expressivity hierarchy governed by the attention type under specific numerical semantics.

Franz Nowak, Ryan Cotterell, Reda Boumasmoud · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.