BRIDGE is presented, a structured prompting framework that decomposes verification into three interconnected domains: Code (implementations), Specifications (formal intent), and Theorem State-ments (constructive correctness claims), and elicits domain-specific intermediate reasoning to connect them.
Large language models (LLMs) have substantially improved code generation, yet achieving strong functional correctness remains difficult, especially for heterogeneous programming tasks where a single prompting strategy and a single directly generated output are often insufficient. In this paper, we present RAV, a lightweight and modular framework that improves code generation with a fixed backbone model through three coordinated stages: Route, which applies task-aware prompt routing before generation; Align, which reduces the mismatch between fine-tuning prompts and inference-time prompts through aligned LoRA adaptation; and Verify, which selects the final output by executing multiple candidates against visible public tests. We evaluate RAV on the MBPP benchmark under both the sanitized and full settings. The complete RAV pipeline achieves the best performance among all evaluated configurations, reaching 0.8911 on MBPP Sanitized and 0.8520 on MBPP Full. Compared with the base model, these results represent improvements of 6.35 and 9.92 percentage points, respectively. Component-wise ablation experiments further show that task-aware routing and aligned adaptation become substantially more effective when combined with execution-based verification. Additional robustness and contamination analyses support the reliability of the observed improvements. Overall, the results indicate that functional correctness in code generation can be meaningfully improved without modifying the backbone architecture, by jointly optimizing how tasks are prompted, how the model is adapted, and how final outputs are selected.
Large language models can generate register-transfer-level (RTL) designs directly from natural language specifications. Their failures, however, arise mostly from understanding rather than coding \cite{zhang2026understanding, qiu2025towards}. A specification is informal and ambiguous, the model's interpretation stays implicit, and every misreading is committed silently into Verilog, where only simulation can expose it. Intermediate representations make the interpretation partly explicit, yet existing works don't verify the interpretation against the specification, and repair simulation failures at the code level regardless of where the misreading originated. VeriRefine instead treats specification refinement as a verifiable stage of RTL generation. It progressively refines the prose specification into an explicit, schema-constrained account of design intent, expressed as per-signal Abstract Signal Transition Functions (ASTFs) that commit each signal's logic style, clock domain, and reset behavior before any code exists and ground every behavior in a verbatim specification sentence. The refined specification then passes a five-layer audit spanning soundness, completeness, consistency, FSM integrity, and core RTL design rules, so interpretation errors are repaired at the representation level before any Verilog is generated. Once code is generated, each simulation failure is classified as an understanding error or a coding error and routed back to the corresponding stage for targeted repair. Because every signal's hardware class is fixed during refinement, synthesizability becomes a structural property of the pipeline rather than a post-hoc check. With Claude Sonnet 4.6, VeriRefine reaches 94.0\% functional correctness on RTLLM v2.0 and 98.1\% on VerilogEval-Human v2.
Xiangfei Kong, Tasnim Tabassum, Marwan Abdelwahab et al.· 0 citations
Large Language Models (LLMs) have transformed software engineering by automating code generation, yet they frequently produce code that is syntactically correct but behaviorally incorrect—failing to respect repository conventions, misapplying practices across contexts, or violating semantic properties. These failures come from a fundamental limitation: current LLMs learn syntactic patterns but lack behavioral understanding of how code executes in context. This work addresses three interconnected challenges in building behaviorally-grounded code generation: (1) retrieving not just what symbols are, but how they behave through usage patterns; (2) adapting practices across contexts by separating behavioral intent from implementation details; and (3) enforcing semantic properties during generation without prohibitive runtime costs. Preliminary results demonstrate substantial improvements: 48.2% exact match on repository completion (vs. 29.05% baseline) and 100% conformance on cross-framework practice transfer (vs. 0% baseline). This work aims to transform LLMs from pattern-matching systems into behaviorally-aware generation system that understand what code does, not just what it looks like.
Large Language Models (LLMs) show promise for synthesizing software directly from natural-language problem descriptions. However, LLM-based code synthesis remains unreliable: models may hallucinate features, generated tests and code may diverge, and repairs often require manual effort. We present a test-driven pipeline that extracts functional requirements (FR) from a problem description, resolves dependencies, maps them into a modular Model-View-Controller (MVC) structure, and generates tests before code, followed by bounded, execution-driven refinement. We evaluate execution reliability with Pass@1 and refine@k, and assess the faithfulness of FR extraction using NLI-based entailment.
Wasay Mohammed Abdul, Ragib Shahariar Ayon, Shibbir Ahmed et al.· SIGSOFT FSE Companion· 0 citations
Unlike natural-language specifications, executable formal specifications provide machine-checkable constraints for verifying, debugging, and repairing code. However, writing such specifications is labor-intensive, and existing LLM-based methods mainly infer whole-program pre/postconditions, missing the intermediate semantic commitments that programmers rely on when reasoning about an algorithm. Our study further shows that prompting current CodeLLMs often produces executable assertions that are syntactically invalid, trivial, or too weak to reject behavior-changing faults. In this paper, we study executable checkpoint specification generation, where assertions are inserted at meaningful internal program points to describe expected intermediate states. We introduce SpecCoder, a verification-guided CodeLLM training framework that learns from validated reference programs, behavior-changing mutants, and multi-turn specification-refinement traces. SpecCoder selects specifications that hold on correct executions while rejecting faulty executions, turning specifications from passive annotations into executable evidence. To evaluate this setting, we introduce HumanExec, a benchmark built from recent Codeforces competitive programming problems with test suites, reference solutions, and human buggy submissions, supporting three tasks: specification generation, program correctness checking, and program repair. Experiments on HumanExec show that SpecCoder substantially improves checkpoint-specification quality over base CodeLLMs. Across Qwen2.5-Coder models, SpecCoder improves inline-specification correctness by up to 55.8%, completeness by up to 358.1%, and executable assertion validity by up to 26.6%. These gains further translate to downstream correctness reasoning and repair, showing that executable checkpoints provide fine-grained evidence for reliable verification.
Minh Le-Anh, Cuong Chi Le, Tien N. Nguyen· 0 citations
LLMs have made substantial progress on automated code generation from natural-language descriptions of desired behavior (intent). Most existing methods improve generated programs through execution-guided code refinement: they generate a candidate solution, execute it, and patch the implementation using feedback, while leaving the underlying specification unchanged. This workflow implicitly assumes that the LLM's understanding of the intent is already correct and complete. In practice, however, intents are often ambiguous or underspecified. As a result, even a capable model may produce a correct implementation of the wrong intent, making specification mismatch a central bottleneck. This paper presents BeSpec, a behavioral model-based approach to specification alignment. BeSpec treats the task description as partial evidence about the intended behavior of the correct program. It first builds an explicit behavioral model, which are checkable properties that valid outputs must satisfy. BeSpec then generates candidate programs, executes them on probe inputs, and compares their observed behavior with the predicted behaviors. When observed behavior does not match the predicted behaviors, BeSpec either refines the specification or rejects the candidate program. We evaluate BeSpec with three LLMs on four benchmarks: CodeContests, xCodeEval, APPS, and the contamination-free LiveCodeBench. Against nine baselines, BeSpec achieves the highest Pass@1 and average pass rate across all settings, improving average Pass@1 over the strongest baseline by 8.1%--25.3% relative across the three LLMs. A failure analysis shows that after alignment, most remaining errors stem from algorithmic difficulty rather than misunderstood specifications, while ablation studies confirm that each major component of BeSpec contributes positively.
Qinghua Xu, Guancheng Wang, Boxi Yu et al.· 0 citations