Skip to content

AssumptionMiner: Extracting, Tracing, and Revising Implicit Assumptions in LLM Code Generation

Jul 2026 · arXiv.org · Vol abs/2607.22898 · 0 citations · 57 references
Computer Science

TL;DR

AssumptionMiner is presented, a framework that makes implicit assumptions a first-class artifact of LLM-based code generation, and demonstrates that making assumptions explicit improves the transparency and controllability of LLM-based code generation.

Abstract

Large language models (LLMs) generate code from natural-language prompts, yet real-world prompts rarely provide complete specifications. When prompts leave input formats, error handling, or design decisions unspecified, LLMs fill these gaps with implicit assumptions that shape the generated code's behavior and correctness. Because these assumptions remain hidden, generated code may satisfy tests while violating developer intent. We present AssumptionMiner, a framework that makes implicit assumptions a first-class artifact of LLM-based code generation. In addition to code, AssumptionMiner produces an explicit assumption layer, a structured representation of inferred constraints and design decisions that developers can inspect, confirm, or revise. An AST-based dependency graph enables targeted regeneration of only the code affected by a revised assumption. We also introduce a benchmark of 180 ambiguous programming tasks with 676 annotated assumptions, including a human-verified subset for evaluating code localization. We evaluate assumption extraction, code localization, and assumption-guided regeneration. Across open-source LLMs, a confidence-weighted ensemble achieves an F1 score of 0.816 for assumption extraction, improving on the strongest offline baseline by 3.6x. On the human-verified localization benchmark, AST-guided localization identifies more precise code regions than keyword-based and whole-file baselines. During assumption revision, targeted regeneration modifies less code than non-targeted alternatives while exposing challenges in handling cascading edits. These results demonstrate that making assumptions explicit improves the transparency and controllability of LLM-based code generation.

View source

Similar papers

Book Open access Oct 2026

A Transformation-Based Benchmark for Evaluating the Robustness of LLMs in Generating OCL

Large Language Models (LLMs) have shown promising performance in generating Object Constraint Language (OCL) constraints from natural language specifications. However, existing evaluations rely on publicly available UML models, which may overestimate generalization due to potential data leakage and reliance on recurrin...

Hamza Attarwala, Moataz Chouchen, Omar Alam et al. · 0 citations
#artificial intelligence Book Sep 2026

Predicting Program Exit Code with LLMs and Programming Language Semantics

This work evaluates open-source coding LLMs under two semantic formalisms and two semantic shifts across Human-Written, LLM-Translated, and Fuzzer-Generated program splits, showing that LLMs lean on pre-training priors rather than systematically applying the given rules.

Lara Marinov, Aditya Thimmaiah, Jayanth Srinivasa et al. · 0 citations
#machine learning Review Sep 2026

Robustness of LLM-Generated SystemVerilog Assertions to Semantics-Preserving RTL Transformations

Large language models (LLMs) are increasingly being explored for automating SystemVerilog Assertion (SVA) generation, yet most evaluations report correctness on a single syntactic representation of an input. Such point accuracy does not reveal whether a model's correct output is stable when the same RTL behavior is wri...

Fnu Aditi · 1 citation
Jul 2026

VClare: Resolving Imperfect Specifications in LLM-Based Verilog Generation

The first systematic study of imperfect specifications is presented and an automated framework to repair them to enhance the quality of resulting Verilog design is proposed, demonstrating the capabilities of specification repair by {VClare} as well as further potential of LLMs in front-end hardware design.

Zhuorui Zhao, Bing Li, Yu Li et al. · 0 citations
Preprint Aug 2026

Unreliable in Practice? A Comprehensive Study of Errors in LLM-Generated Code

It is observed that generated code often omits basic input validation or memory-safety checks, which can lead to overflows, resource exhaustion, or other reliability/security issues, and even the largest models frequently make simple mistakes.

Rodrigo Pato Nogueira, Marco Vieira, João R. Campos · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.