The learned input-output behavior of GLU-based neurons in large language models (LLMs) is analyzed and it is found that weakening neurons display surprising behavior: even though there are few, they activate often and have a large influence on model behavior.
Sebastian Gerstner, Hilal AlQuabeh, Kentaro Inui et al.· 0 citations
Neuron steering reinforces dialect in some varieties when the prompt is already dialectal but cannot induce it from MSA prompts, whereas vector steering succeeds in both settings, and Arabic dialects are therefore steerable mainly through distributed rather than localized representations.
K. Elozeiri, Mervat T. Abassy, Omar Kallas et al.· arXiv.org· 0 citations
A detector-guided self-correction method is proposed that selectively re-prompts the model only when an internal error detector predicts failure, and a simple inference-time intervention improves counting accuracy by up to 15.6 absolute percentage points.
Ahmed Oumar El-Shangiti, Abzal Nurgazy, Hilal AlQuabeh et al.· arXiv.org· 1 citation
ProbeDrift is introduced, a systematic evaluation framework for supervised uncertainty probes covering a wide range of OOD settings across models, tasks, and distributional shifts, and it is argued that robust uncertainty estimation requires robust evaluation.
Joe Stacey, Hadas Orgad, Kentaro Inui et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.