Selecting and Combining Large Language Models in Scalable Code Clone Detection

Abstract

Source code clones pose risks ranging from intellectual property violations to unintended vulnerabilities. Effective and efficient scalable clone detection, especially for diverged clones, remains challenging. Large language models (LLMs) have recently been applied to clone detection tasks. However, the rapid emergence of LLMs raises questions about optimal model selection and potential LLM-ensemble efficacy. This paper addresses the first question by identifying 76 LLMs and filtering them down to suitable candidates for large-scale clone detection via an LLM-encoder framework known as SSCD. The candidates were evaluated on two public, industry-defined datasets, BigCloneBench, and a commercial, large-scale dataset. No uniformly ’best-LLM’ emerged, though CodeT5+ 110M, CuBERT and SPTCode were top-performers. Regression analysis suggests that embedding size, tokenizer vocabulary, and training dataset characteristics are associated with clone detection performance. To address the second question, this paper explores the ensembling of selected LLMs to improve effectiveness. Results suggest the importance of score normalization and favoring ensembling methods like maximum or sum over averaging. Also, findings indicate that ensembling approaches can be statistically significant and effective on larger datasets, where the best-performing ensemble improved performance by 37% over its individual LLMs on the commercial large-scale code.