Removing the NEEDLE in the Haystack: Backdoor Removal in LLMs via Weight Orthogonalisation
This work proposes NEEDLE, a training-free method for targeted backdoor removal that achieves the lowest mean Attack Success Rate (ASR) among the evaluated defences, including 0% on challenging code injection attacks, while resulting in the lowest KL divergence and minimal changes in capability and safety.