The use of AI agents for automatic code generation has become increasingly common in software development. However, concerns remain about the quality of the generated code, including aspects of maintainability, readability, and long-term evolution. This study compares the structural quality of code produced by three widely adopted vibe coding tools --- Lovable, v0, and Replit --- starting from a single generation prompt. We generate three independent projects per tool, totalling nine web applications, and submit them to static analysis with SonarQube. We collect metrics such as the number of issues, severity distribution, estimated remediation effort, cyclomatic and cognitive complexity, and code duplication. Preliminary results show that the tools exhibit distinct qualitative profiles: Lovable concentrates issues of lower severity but presents a substantially higher density of code smells per KLOC, while v0 and Replit produce more code with more aggressive severity profiles. These findings suggest that choosing between vibe coding tools involves structural trade-offs that go beyond perceived productivity.
The emergence of large language models (LLMs) has significantly impacted the automation of software development tasks, including the generation of mobile application code. This research investigates how LLMs, such as ChatGPT, can support the creation of accessible native Android applications. Specifically, it evaluates whether the generated code aligns with established accessibility guidelines, considering variations in screen layouts, prompt styles, and interface design methods. Four empirical studies were carried out to assess accessibility in seven different types of mobile UI. The first study assessed screens built with various layout techniques, while the second focused on Jetpack Compose and compared outputs from multiple LLMs (e.g., Sabiá, GPT). The third experiment explored the influence of English-language prompts on accessibility outcomes, and the fourth involved an evaluation of Github Copilot. In all studies, 702 accessibility-related issues were identified. Jetpack Compose consistently led to more accessible interfaces compared to other layout options, and prompts in English resulted in fewer errors. Surprisingly, prompts that explicitly asked for accessibility often introduced more problems, suggesting that current models struggle to accurately interpret and implement accessibility directives. These findings underscore the need for improved prompt engineering and more robust LLM-generated code to improve accessibility in AI-assisted mobile development.
Daniel Mesquita Feijó Rabelo, Júlia Holanda Muniz, K. Gama et al.· Universal Access in the Info...· 0 citations