Shuo-Jin Yang

CyFA: Linear Sequence Modeling with Relative-Time-Partitioned Memory

Linear RNNs offer linear-time sequence processing and constant-memory decoding, but their fixed-size recurrent states must accommodate all past key--value associations. Existing forgetting mechanisms and Delta Rule updates reduce interference by selectively clearing or correcting the state, yet earlier associations can...