Applications of Generative AI for Advancing Scientific Content Production, Evaluation and Dissemination

Abstract

Generative AI is becoming part of how scientific content is being produced, evaluated and communicated. Large language models (LLMs) can write science news, simulate readers, retrieve evidence, generate scientific hypotheses, write code, and execute research workflows. These capabilities create opportunities for accelerating scientific discovery and broadening access to scientific information, but they also introduce risks related to hallucination and trustworthiness. This doctoral dissertation studies how generative AI can support scientific content production and dissemination, while preserving trust, comprehension, and replicability. The completed work includes three components: (1) detecting LLM-generated science news through SANews and GFS, (2) evaluating science communication through reader learning using KnowledgeGain and LLMSim, and (3) benchmarking LLM agents for research replication with ReplicatorBench and ReplicatorAgent. The proposed work extends these directions toward learning-optimized science news generation, MESH for multimodal evidence-grounded hypothesis generation, and MERIT, a simulation-grounded framework for evaluating autonomous AI co-scientists and hypothesis testing in computational materials discovery. The overarching goal is to develop evaluation methods and AI systems that make scientific outputs not only fluent and scalable, but also grounded, useful, and interpretable.