This vision reframes AI code review from automated commenting to human-AI sensemaking before integration, and outlines a research agenda for studying review conversations, designing conversational AI review capabilities, and evaluating their impact on software evolution and maintenance.
Abstract
AI-based code review tools increasingly promise to help developers inspect pull requests, identify defects, and improve code quality. Yet most current approaches frame code review as a one-shot commenting task: given a diff, the system produces warnings or suggestions. This framing overlooks a central property of modern code review: review is a conversation. Human reviewers do not merely comment on code; they ask questions, explain expectations, negotiate design trade-offs, request evidence, transfer project knowledge, document rationale, and collectively decide whether a change is good enough to integrate. In this vision paper, we argue for conversational AI review assistants: systems that participate in code review as interactive partners rather than static comment generators. Such assistants should identify when conversation is needed, ask grounded questions, respond to developer explanations, summarize unresolved issues, help capture rationale, and know when to abstain or escalate to human reviewers. Such a paradigm shift requires novel evaluation methodologies as well. We outline a research agenda for studying review conversations, designing conversational AI review capabilities, and evaluating their impact on software evolution and maintenance. Our vision reframes AI code review from automated commenting to human-AI sensemaking before integration.
ARCTIC, an AI-powered Code Critique system that reframes code review around three capabilities: intent prediction, which infers why a change was made from conversation logs and metadata; drift detection, which measures divergence between the developer's intent and the agent's output via backtranslation; and code spotli...
C. Maddila, Mashrur Rashik, E. Khan et al.· arXiv.org· 0 citations
A large-scale empirical characterization of AI-assisted OSS contribution is provided and practical insights for designing and governing responsible vibe-coding practices in open-source development are offered.
The first large-scale empirical study on the resolution of agent-generated code review comments is presented, revealing that the presence of an inline code suggestion is the strongest predictor of comment resolution, while lengthy and complex comments are less likely to be acted upon.
Shamse Tasnim Cynthia, Ratnadira Widyasari, Banani Roy et al.· arXiv.org· 0 citations
Background: Large language models (LLMs) are increasingly used to automate code review, but the reasoning behind their decisions remains hard to understand. Developers struggle to assess the validity of LLM-generated reviews, making it difficult to gauge how much trust to place in them. The role of Explainable AI (XAI)...
Zhenhan Gao, Marvin Muñoz Barón, Umm E. Habiba et al.· arXiv.org· 0 citations
A vision for inclusive, human-centered AI-assisted code review systems that adapt feedback to developers' problem-solving preferences is outlined, highlighting the need to balance personalization with clarity and trust.
Shamse Tasnim Cynthia, Ratnadira Widyasari, Banani Roy et al.· 0 citations
The results show that agent-involved collaboration patterns, especially reviews initiated by AI agents or involving multiple AI agents, are associated with faster review decisions under Gradual AI Adoption and Rapid AI Agent Adoption, but these efficiency gains do not translate into better review quality.
Suzhen Zhong, Shayan Noei, Bram Adams et al.· 2 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.