Andrew Delarosa

Androw of Aces: Mitigating Autoregressive Arithmetic Drift in Large Language Models via Canonical Pre-Sorting and Key-Value Bound Binary Prefix-Sum Scratchpads

Autoregressive Large Language Models (LLMs) frequently fail at multi-variable arithmetic aggregation when operating in-context without external code execution. Bounded by circuit complexity class $\mathsf{TC}^0$, standard transformers struggle with sequential carry states across lists of length $N > 3$, suffering from...