This work proposes Jacobian Scopes, a suite of gradient-based, token level causal attribution methods for interpreting LLM predictions that reveal implicit political biases, uncover word- and phrase-level translation strategies, and shed light on recently debated mechanisms underlying in-context time-series forecasting...
To-Ni J. B. Liu, Baran Zadeoglu, Gurbir Arora et al.· 0 citations
This work studies activations from ensembles of prompts that require inferring relationships between three tokens corresponding to a cyclic concept to correctly predict the next token and finds a consistent layerwise progression in how the joint dependence among the tokens is geometrically organized and causally used.
Gurbir Arora, To-Ni J. B. Liu, Jiajun Bao et al.· arXiv.org· 0 citations
The direction of ignorance is causally active: raising or lowering $\lambda$ at the final prediction state steers the prediction toward or away from the unigram prior in KL divergence, with larger models generally exhibiting lower prior reliance in the high-context limit.
To-Ni J. B. Liu, Jiajun Bao, Yizhou Liu et al.· 0 citations
Puzzling internal representations as a practical tool for calibrating, auditing, and triaging language model forecasters and reasoning models more broadly is established.
Raphael Sarfati, P. Tiwari, S. Boppana et al.· arXiv.org· 2 citations· ⚡1
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.