Symmetry-Transport Hypernetworks: Efficient On-Demand Weights by Separating Task Geometry from Conditional Parameter Generation

Abstract

A hypernetwork can generate task-specific model weights from a compact representation of the task, rather than learning a separate set of weights for every transformed version. In this work, invariant task coordinates are used to generate canonical weights, which are then transformed according to the known task symmetry. For planar tasks, a rank-four, threshold-conditioned generator produces all 33 parameters of an eight-neuron classifier using 173 learned parameters, compared with 477 in a matched dense hypernetwork. The construction guarantees exact rotational consistency. A fused predictor computes the same function by rotating the inputs, avoiding the need to generate transformed weights explicitly. Across 13 configurations, three synthetic task families, and ten training seeds, the selected model achieves 98.67% accuracy on unseen curved-task contexts, compared with 96.80% for the matched dense baseline. For 2,048 dynamic queries, fused inference takes 1.01 ms, compared with 2.24 ms for vectorized dense inference and 264.62 ms for a scalar reference implementation. The results also reveal important limitations. A symmetry-breaking control reverses the accuracy advantage, while an MNIST pair diagnostic finds that the generated model is both less accurate and slower than independent baselines. All experiments are synthetic, so these findings do not establish whether the approach will work at the scale of large language or vision models.