Skip to content
#software testing Review Open access

From Code Author to Code Reviewer: A Systematic Review of Practitioner Experience with Generative AI in Software Engineering

Oct 2026 · Journal of Information Systems and Informatics · 0 citations
Software Engineering Research

TL;DR

This review examined 58 empirical studies published from 2024 to early 2026 on GitHub Copilot, ChatGPT, Claude, and Gemini using PRISMA-guided selection from five digital libraries to address productivity, role shifts, code quality, adoption behaviour, and evidence gaps, and foundings were synthesised into the AI Integration Consequence Model.

Abstract

Practitioner use of generative AI in software engineering has progressed faster than the evidence assessing its effects. This review examined 58 empirical studies published from 2024 to early 2026 on GitHub Copilot, ChatGPT, Claude, and Gemini. Using PRISMA-guided selection from five digital libraries, the studies addressed productivity, role shifts, code quality, adoption behaviour, and evidence gaps. Findings were synthesised into the AI Integration Consequence Model, a conceptual four-layer framework linking adoption conditions, role shifts, quality exposure, and organisational consequences. Evidence indicates that developers increasingly move from code production to code review, as more than half of AI-generated code is discarded after evaluation. Generative AI improves efficiency for boilerplate code, documentation, and unit-test scaffolding, but provides limited benefits for architectural decisions and enterprise debugging. A meta-analysis of 47 studies reported a 45.6-point gap between self-reported and objectively measured productivity. AI-generated code also showed exploitable vulnerabilities at estimated rates of 40–50%, varying by task, tool version, programming language, and measurement method. Social influence strongly predicted adoption intent, while concerns about peer judgement encouraged undisclosed use. Experienced engineers generally received an AI Boost, whereas junior engineers faced an AI Drag. Structured mentorship reduced this gap in one 14-month cohort study. Key evidence gaps include cross-cultural adoption, longitudinal skill development, and unreplicated gender disparities.

Read PDF

Similar papers

Review Open access Sep 2026

Rethinking Code Review in the Age of Generative AI: A Conceptual Framework for Layered Software Verification

It is concluded that generative AI does not eliminate the need for human code review but requires a redesign of its role and proposes a four-layer verification architecture comprising deterministic automated verification, AI-assisted contextual verification, independent human technical review, and human accountability...

Usman Karim · 0 citations
Book Open access Nov 2026

Towards Comparative Comprehension in CS1: An Exploratory Study using Structured Code Comparisons

This work explores how structured code comparison activities support comparative comprehension in CS1 students transitioning from Snap! to Python, and highlights both the challenges and opportunities of supporting comparative comprehension through structured comparison activities, particularly in terms of timing, scaff...

Azeeza Eagal, John-Paul Ore, Veronica Cateté · 0 citations
Preprint Oct 2026

When Code Gets Cheap, Evaluation Gets Scarce: An Experience Report on AI-Assisted Agile Software Engineering

Coding assistants are widely reported to speed up code production; most accounts stop there. We report the end-to-end construction and deployment of a compliance-critical, human-in-the-loop exam-grading system, built by a single developer who wrote no code by hand, protected by a gated process from stakeholder persona...

Barbara Weber, Olaf Zimmermann, Marco Franceschetti · 0 citations
#software testing Review Open access Oct 2026

Automated Testing and Its Impact on Software Development Quality and Maintainability: A Focused Structured Review

A transition from code-level defect metrics toward process- and system-level maintainability concerns is highlighted, and five research gaps are identified: separating testing practice from testing metrics, context-aware empirical design, standardized measurement and reporting, longitudinal maintainability tracking, an...

Kashodhan Prinkal, S. Degadwala · 0 citations
Review Oct 2026

When AI Can Build It, What Should We Assess? Rethinking Evidence of Engineering Competence

Generative AI can produce code, tests, documentation, and technical explanations that resemble artefacts traditionally used as evidence of student competence, weakening the inference from artefact quality to the engineering capability behind it. We report the redesign and first deployment of a 13-week, AI-permissive, s...

Ali Shakiba · 0 citations
Preprint Oct 2026

Does AI-Assisted Development Homogenize Software? An Empirical Study of Code Similarity, Quality, and Evolution

AI-assisted software development is evolving from interactive LLM assistance toward agentic systems that autonomously perform software engineering tasks, yet the effects of this shift remain insufficiently understood. We present an empirical study of 50 long-lived Java and Python open-source projects across three eras:...

Luciano Marchezan, Romain Lefeuvre, Kévin Delcourt et al. · 0 citations

Related blog posts

Microsoft Research Blog Nov 12, 2026

Microsoft Research Focus – Edition #98

What's new in research at Microsoft? Host Anson Ho explores the latest, including AI-powered universal translation, a breakthrough in a 31-year-old mathematics challenge, and advances making AI coding agents faster and more efficient.  The post Microsoft Research Focus – Edition #98 appeared first on Microsoft Research.

Microsoft Research Blog Nov 5, 2026

Formally verifying agent-written full-stack web apps with PDD Kit

AI agents can generate code at a scale that’s hard for humans to review line-by-line. PDD Kit helps ensure the code is doing what the developer intended by creating formal specifications and testing against them. It converts developer intent into a Verus specification, proves AI-generated code meets it for every input, and uses tests where proofs aren't practical. Developers review intent, not every line. The post Formally verifying agent-written full-stack web apps with PDD Kit appeared first o…

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.