Test-Free Fault Localization Through Deep Source Code Analysis Using Encoder-Based Language Models

Abstract

Fault Localization (FL) has evolved with the emergence of the Transformer architecture. Traditionally, FL relied on generating a ranking of suspiciousness scores for each code unit using information obtained during program execution from a set of test cases. However, Language Models trained with source code have enabled approaches that avoid the need for test suites, which are, sometimes, difficult or time-consuming to obtain. In this work, we present a refined training strategy tested on DeBERTa-v3-base, leveraging an encoder-based approach, to address two key challenges in FL: (1) the accurate classification of lines of code according to their suspiciousness scores, and (2) the class imbalance inherent in this type of problem. To this end, we extract the last hidden state corresponding to the embeddings of the new line tokens to generate a suspiciousness score for each line. This score then classifies each line as defective or not. The evaluation of our approach on the ConDefects dataset, repeated over ten random seeds, shows promising results compared to existing approaches, with ROC AUC gains up to 0.27 in cross-dataset comparisons. Thereby, we offer a competitive baseline for future test-free fault localization research while using language models that are approximately 200 times smaller than those used in prior work.