Generative AI use in Statistical Research: A Literature Review and Code Generation Case Study
TL;DR
GenAI can function as a research tool, but not as a substitute for methodological expertise, and has potential to increase efficiency of tasks which take advantage of its search and summarization abilities, as well as basic code debugging and algorithm formation.
Abstract
Generative artificial intelligence (GenAI) is a large language model (LLM) that has the ability to generate media based on user-provided prompts. Given the demonstrated capabilities of models such as ChatGPT in information synthesis and programming, there is growing interest in their potential role within the research process. However, little work has evaluated recent GenAI models for research tasks in the domain of statistical research. This case study examines GenAI as a tool for developing a literature review and translating methodology from academic papers into code, for the topic of dynamic treatment regime (DTR) estimation via the dynamic weighted ordinary least squares (dWOLS) approach. Specifically, we utilize ChatGPT-5 and ScholarAI (Sept-Nov 2025 release) in the processes of identifying relevant sources for the literature review, creating summaries of papers, identifying gaps in research, and R code generation to implement methodology. Our findings show that current GenAI models lack the depth and contextual understanding required to accomplish these tasks without careful prompting and supervision of a knowledgeable researcher. Nonetheless, GenAI has potential to increase efficiency of tasks which take advantage of its search and summarization abilities, as well as basic code debugging and algorithm formation. We demonstrate that under a knowledgeable guide, GenAI can function as a research tool, but not as a substitute for methodological expertise.