Learned Structure Shows Up Earlier in the Task's Own Basis: A Controlled Look at a Synchronisation Precursor in Grokking

Abstract

Analyses of grokking usually read hidden activations in one basis, most often the Fourier basis modulo $p$. We ask how much the answer to "has the network organised yet?" depends on that choice. The measure FSD$_{\rm gen}$ averages each unit's activation over inputs with the same answer and counts how many units share the same strongest irreducible representation. For modular multiplication we read one answer-averaged profile in three orders: discrete-logarithm order, in which the characters of the multiplicative group become Fourier modes; natural order; and random orders. Across 60 networks at $p\in\{59,97,113\}$, the discrete-log reading reached a fixed threshold before both other readings in 60 of 60 networks (59 of 60 when it had to hold 500 steps). Because untrained networks already score higher in discrete-log order, we also required an equal rise above each reading's untrained level; the discrete-log reading was still first in at least 16 of 20 networks at every prime. At $p=59$ and $p=97$ this happened during the memorisation plateau in 35 of 39 networks; at $p=113$ in 12 of 20. Memorisation preceded grokking in all 179 runs that grokked, by more than any synchronisation signal on average, so a lead over grokking is not by itself evidence of a detected circuit. Synchronisation time is only weakly related to grokking time (pooled $\rho=0.18$). On six finite groups, read with the same measure in every basis, the irreducible-representation reading came before random element orders on all six; with equal-rise thresholds this held in every case on four groups, in 7 of 10 on $\mathbb{Z}_5\times\mathbb{Z}_5$ and in none on $D_{16}$, where about half the profile power lies in one-dimensional representations the measure leaves out. We also correct a conjugate-pair artefact that roughly halves the measure on abelian groups. This version withdraws the causal and predictive claims of v1.