Exploring the Effect of Multiple Languages on Code Suggestion Using ChatGPT 5.2

Abstract

: Artificial Intelligence (AI)-based code suggestion tools are becoming increasingly relevant in the software development community. While several studies analyse the quality and security of these tools, relatively few systematic studies have investigated how different input text languages can affect the effectiveness of code generation models. Similar research has been conducted with other tools, addressing multiple programming languages and different languages and challenge platforms. However, it is important to expand this knowledge base by considering additional scenarios. In this study, we analyse the performance of ChatGPT 5.2 in generating Python solutions for 70 programming problems extracted from the Beecrowd repository, considering three languages: Portuguese, English and Spanish. These 70 challenges were selected to encompass a range of difficulty levels for mathematical problems. Each question was submitted to the model five times in each language and the accuracy rate was evaluated using the platform’s validation tests. It was observed that Portuguese prompts generally showed a slight advantage in terms of the accuracy rate compared to prompts in other languages. However, the analysis of variance (ANOVA) indicates that this variation was not significant at the 5% level. The aim of this article is to extend current knowledge about the impact of languages on code generation by demonstrating that the language used in the problem description may or may not affect the ability of AI models to provide correct solutions. Finally, this article discusses the influence of potential linguistic biases, as well as opportunities for future research to address other languages, different problem modalities and various AI tools.