An Empirical Study on the Transferability of Transformer-Based Models for Software Vulnerability Detection

Abstract

Despite the dominance of Transformer-based models in software vulnerability detection, the extent to which their learned security logic generalizes across different programming languages remains a critical open question. To address this, we propose a comprehensive evaluation framework organized into three phases spanning five distinct experimental scenarios, aiming to rigorously dissect model behaviors from input sensitivity and baseline performance to cross-lingual transferability and decision transparency. We benchmark four representative models (CodeBERT, GraphCodeBERT, UniXcoder, and CodeT5+) using a curated Polyglot Injection Dataset (C/C++ and Java). Our empirical results reveal that extending input context from 128 to 512 tokens yields negligible gains, suggesting a heavy reliance on localized features. While we observe a distinct transfer asymmetry favoring C/C++ sources, the Encoder-Decoder architecture (CodeT5+) demonstrates superior cross-lingual generalization. Crucially, high structural robustness against semantic transformations, combined with quantitative XAI agreement analysis, confirms that these models capture genuine vulnerability logic. These findings suggest that the primary barrier to cross-lingual transfer is vocabulary mismatch rather than a failure in reasoning capabilities.