Hyphae Research Foundation

The Missing Medium: A Transactional Residual Stream for Transformer Training

Modern training treats the transformer residual stream as an anonymous accumulator: any kernel may execute h += \delta, and no record of the write survives the step. Training observes the consequences of internal writes — loss curves, evaluations, activation probes — never the writes themselves. We introduce hytorch, a...