Trace as State: Reasoning Traces as Conditional States for Long-Context Transformers
Results show that placing traces before the context can improve long-context reasoning while retaining the causal transformer structure, and Trace as State outperforms Trace Append in 26 of 27 reported combinations of model, task, and metric.
Xu Zou, Jie Tang
· 1 citation