Skip to content

SpecMine: A Large-Scale Corpus of Spec-Driven Development Artifacts

Aug 2026 · 0 citations · 4 references
Computer Science

TL;DR

SpecMine lets the community study, for the first time, how software is specified in the age of AI agents through two censuses: a broad census of spec.md files and a census-wide index of typed references.

Abstract

Spec-Driven Development (SDD) is a fast-emerging practice in which a structured natural-language specification, written by a developer, or (more often) drafted by an AI tool and then curated by the developer, drives an AI coding agent's implementation. A wave of tooling (GitHub Spec Kit [3], OpenSpec [4], AWS Kiro [5], and dozens of others) has appeared since 2025, yet the artifacts these tools produce have never been studied at scale. We present SpecMine, a corpus that captures SDD in public GitHub repositories through two censuses: a broad census of spec.md/specs.md files covering most tools (470,795 files across 73,030 repositories, attributed to 17 named tools), and a Kiro census of its distinct requirements/design/tasks layout (98,574 files across 12,910 repositories). Each spec is enriched with full repository metadata, complete commit history, and parsed document structure. How a spec becomes code is itself an open question, so for 11 tools we sweep every pull request that touches a spec in their repositories with at least ten stars, capturing 5,992 such PRs across 581 repositories with their changesets. That makes the simplest workflow, spec and implementation changing together in one PR, directly observable, and a census-wide index of 2,421,323 typed references (1.28M to code files, 863k to sibling documents, 152k to PRs, 62k refs, 43k branches, 22k issues) gives a second, independent link from spec to code. SpecMine lets the community study, for the first time, how software is specified in the age of AI agents.

View source

Similar papers

Conference Aug 2026

Generating Behavior-Driven Development Artifacts

Behavior-Driven Development (BDD) scenarios coexist with large volumes of semi-structured records (e.g., documentation, issues, informal feature descriptions), and keeping the two synchronized is laborintensive. We present a study of bidirectional generation 11https://github.com/Artin-Biniek/Submission-Project between...

A. Biniek, Nafiseh Kahani · 0 citations
#artificial intelligence Review Sep 2026

Can LLMs Extract Architectural Design Decisions from Source Code Commits? - A Preliminary Exploratory Study

A preliminary study using four LLMs with zeroshot and fewshot prompting on 30 developer-written ADDs from open-source projects finds that the generated ADDs are often too long, implementation-focused, and miss the rationale behind the decision.

Amey Karan, Rudra Dhar, Mohamed Soliman et al. · 0 citations
Preprint Aug 2026

Comparing the Quality of Code Generated by Vibe Coding Tools

This study compares the structural quality of code produced by three widely adopted vibe coding tools --- Lovable, v0, and Replit --- starting from a single generation prompt and suggests that choosing between vibe coding tools involves structural trade-offs that go beyond perceived productivity.

Gustavo Rubin da Mota, Kiev Gama · 0 citations
Preprint Aug 2026

Unreliable in Practice? A Comprehensive Study of Errors in LLM-Generated Code

It is observed that generated code often omits basic input validation or memory-safety checks, which can lead to overflows, resource exhaustion, or other reliability/security issues, and even the largest models frequently make simple mistakes.

Rodrigo Pato Nogueira, Marco Vieira, João R. Campos · 0 citations
Preprint Sep 2026

LLMVul: A Vulnerability-Labeled Dataset of LLM-Generated C/C++ Functions from Real Production Repositories

LLMVul, a vulnerability-labeled dataset of LLM-generated C/C++ functions mined from real production repositories, enables reproducible research on vulnerability detection, security evaluation of LLM-generated code, and analysis of vulnerability patterns in AI-assisted software development.

Mohammad Farhad, Shuvalaxmi Dass · 0 citations

What does AI mean for Open Source?

The recent meteoric rise of LLMs (Large Language Models) and associated tools was largely unexpected and surprising to most. The rapid ascent of this technology has caught many software developers unawares, leaving them suddenly somewhat ignorant, and arguably under-skilled. LLMs, whilst still advancing, have recently...

Adam Retter · 0 citations

Related blog posts

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.