Cross-Model In-Context Neurofeedback Reveals (Limited) Activation Control in LLMs
Probes on activation vectors are widely used to detect misbehavior in language models, but their success relies on the assumption that models cannot control what their activations encode. Humans can learn, through neurofeedback, to control the firing of even single neurons, and recent studies investigated whether this...
Davide Baldelli, Robyn Kozierok, Mirko Bronzi et al.
· 0 citations