Automated Code Review in DevOps Pipelines using Artificial Intelligence

Abstract

In Ensuring fast and precise code review process has become one of the major problems of the modern DevOps world, where software development happens quickly for code reviews to be done manually. In this paper, we introduce a novel automated code review framework named AICR-DevOps, which utilizes a combination of rule-based static analysis, fine-tuned CodeBERT semantic classifier, and large language model using confidence-based aggregation. The framework architecture allows successfully combining the power of traditional program analysis approaches and artificial intelligence reasoning capabilities to achieve high review accuracy with minimized false positives. Together with GitHub Actions, the framework is capable of conducting pull request analysis instantly and constantly adapting to specific coding practices of projects by fine-tuning based on developer feedback using LoRA method. Experimentation with the proposed framework on 12,400 pull requests gathered from 40 Java and Python repositories reached 84.7% precision, 81.3% recall, and an F1-score of 82.9%, with review comments provided in 38 seconds on average. Moreover, a controlled study conducted with 48 developers showed that the acceptance rate of the developers to the framework reached 73.6%.