Fully Unsupervised Cross-Lingual Capabilities Enhancement Based on Language-Specific Neurons

Abstract

Multilingual large language models have shown impressive capabilities in different aspects, but suffer from the performance gaps between dominant languages and low-resource languages. Although fine-tuning methods can be effective solutions, they require high-quality parallel data and substantial computation. To tackle these problems, we propose a training-free intervention framework Cross-Lingual Enhancement via Activation Steering (CLEAN), which improves cross-lingual performance by only using non-parallel multilingual sentences. CLEAN identifies a sparse set of language-specific neurons that exhibit English activation patterns. During the inference of other languages, CLEAN steers these neurons’ activations, effectively allowing low-resource language processing to dynamically leverage the model’s well-trained English parameters without retraining. Extensive experiments show that CLEAN significantly improves cross-lingual accuracy and introduces negligible computation overhead. Additional analyses of neuron identification reliability, layer-wise intervention, semantic alignment, and computational cost further support the effectiveness and efficiency of CLEAN. Evaluations on multiple model backbones, including a Chinese-centric model, further validate the generality of the proposed pivot-language intervention framework.