The results show that specification size alone does not predict implementation quality and that cross-agent transfer can produce substantial agent-dependent degradation, and suggest that specifications in heterogeneous SDD workflows should not automatically be treated as agent-neutral artifacts.
Abstract
This paper investigates cross-agent specification portability using Oracle-to-PostgreSQL migration as a controlled software transformation task. The study combines two experimental stages. First, a specification-first migration pipeline was evaluated on 1,006 PL/SQL files, of which 623 were successfully regenerated and 380 generated scripts executed successfully in PostgreSQL 16. Second, cross-agent experiments were conducted on a dataset of 1,802 Oracle scripts with corresponding PostgreSQL implementations using Amazon Kiro, Google Gemini, and GitHub Copilot, with Claude Code and Cursor included in the initial single-agent evaluation. Native and foreign specifications were assessed using Token F1, exact match, SQL syntax validity, AST exact match, AST mean similarity, and immediate runnability. The results show that specification size alone does not predict implementation quality and that cross-agent transfer can produce substantial agent-dependent degradation. The strongest replicated case occurred when Gemini directly consumed a Kiro-origin specification, producing a Token F1 of 0.035, SQL syntax validity of 2.33%, and AST mean similarity of 0.015. Rewriting substantially improved Gemini in the tested configuration, compression did not provide a universal benefit, and retrieval-augmented ingestion was the only common strategy represented on the per-agent Pareto frontiers of both Gemini and Copilot. The findings suggest that specifications in heterogeneous SDD workflows should not automatically be treated as agent-neutral artifacts and motivate explicit consideration of specification portability, agent-specific interpretation, and retrieval-based access in multi-agent software engineering.
Large Language Models (LLMs) are increasingly used to generate executable software environments from repository artifacts. However, functional executability does not necessarily imply conformity with architectural, security, workflow, and production intent. This study investigates whether a frontier LLM can generate Dockerfiles and Docker Compose configurations for multi-service applications using repository contents without access to developer-authored deployment artifacts. Three heterogeneous repositories combining Python, Node.js, .NET, React, Rust, Java, Redis, PostgreSQL, and MySQL-compatible infrastructure were evaluated using deterministic end-to-end HTTP oracles and manual structural comparison. All three generated environments became functionally operational, although one required a Rust base-image update from version 1.85 to 1.88. The model correctly reconstructed service topology, application ports, infrastructure dependencies, service hostnames, a background worker, hidden proxy configurations, and a file-based Docker secrets mechanism. However, it consistently omitted network segmentation, multi-stage builds, dependency-layer caching, live-reload volumes, production frontend serving, restrictive backend-port policies, and cross-platform build logic. Based on these observations, the study formalizes the distinction between functional correctness and deployment-intent fidelity and analytically derives a minimal explicit deployment specification for information that cannot be reliably inferred from repository artifacts.
Oleg Grynets, Kyrylo Fursov, V. Lyashkevych et al.· 0 citations
Single-repository quality gates are blind to API contract violations across microservice boundaries: a pull request that changes a REST endpoint can pass all local CI checks while silently breaking downstream consumer services in other repositories. This paper presents the K11tech Microservice QA System, a LangGraph-orchestrated agentic pipeline that extends CI/CD quality assurance to distributed microservice ecosystems. The system maintains a cross-repository dependency graph, extracts versioned API contracts from pull requests (OpenAPI 3.x, gRPC Protocol Buffers, GraphQL, and AsyncAPI 2.x), identifies all downstream consumers of changed endpoints, and dispatches parallel ContractComplianceAgents to validate compatibility before merge. A cross-repository HITL gate triggers on impact score thresholds, filing GitHub issues in both provider and consumer repositories when breaking changes are detected. Evaluation across four phases (90 scenarios) achieves 100% precision, recall, and F1 throughout, including external validation on 59 OSS pull requests across four repositories with zero false positives and transitive consumer discovery at depth-2. Multi-model ablation identifies Claude Haiku 4.5 as the optimal backend: 31% faster than Sonnet 4.6 (8.9 s vs. 12.9 s) while maintaining perfect accuracy on ambiguous consumer-impact scenarios. The system is publicly available under the Apache 2.0 licence.
Kavita A. Jadhav· International Journal Of Eng...· 1 citation
Multi-agent systems built on large language models (LLMs) are increasingly deployed for complex tasks requiring autonomous planning, tool use, and inter-agent coordination. However, the non-deterministic nature of LLM outputs and the emergent behavior arising from agent interactions render traditional test oracles ineffective, creating a critical gap in quality assurance for agentic AI. This work introduces MORPHAGENT, a framework designed to address the oracle problem in multi-agent LLM systems through trace-based behavioral analysis. Our contributions are threefold: (1) goal-preservation relations that verify consistent goal achievement under input perturbations, (2) coordination-consistency relations that validate inter-agent delegation and communication patterns under agent substitution and reordering, and (3) tool-use integrity relations that ensure semantic equivalence of tool invocation sequences under prompt paraphrasing. MorphAgent instruments agent execution to capture structured traces comprising planning steps, tool calls, message exchanges, and final outputs, then systematically applies metamorphic transformations and checks behavioral invariants without requiring ground-truth oracles. We evaluate the framework on four multi-agent benchmarks spanning code generation, research synthesis, customer service, and data analysis tasks, encompassing 2,840 source-followup execution pairs across three LLM backends. Results show that MORPHAGENT detects 82.0% of seeded behavioral faults, including 90.3% of coordination failures and 81.7% of goal-deviation faults, while maintaining a false positive rate of 6.1%. The framework uncovers 14 previously unreported behavioral anomalies in established multi-agent frameworks, demonstrating its practical utility for assuring agentic AI reliability. These results suggest that trace-based metamorphic testing can serve as a practical foundation for reliable validation of emerging agentic AI systems.
Gopalakrishnan Marimuthu· International Conference on...· 0 citations
: Integrating Large Language Models (LLMs) into production-grade business systems requires a clear boundary between deterministic application logic and non-deterministic AI behavior. This paper introduces a runtime architecture that extends the BAB (Business Application Builder) framework on Jakarta EE to support agent-augmented applications hosted alongside ontological-driven generated cores. The core of the solution, the AgentRuntimeManager , moves beyond simple API wrappers by providing a robust environment for agent lifecycles, asynchronous event dispatching, and error handling through configurable retry policies. To ensure system stability, we decouple LLM providers (such as OpenAI and Ollama) using an abstraction layer built on LangChain4j, allowing for runtime provider switching without redeploying the core logic. We demonstrate the practical utility of this architecture through a self-healing log-monitoring chain. In this setup, agents intercept high-severity log events and, through a structured analysis service, propose corrective actions. A specialized SqlErrorHealingAgent illustrates how LLM-proposed fixes can be filtered through layered safety constraints, such as the automatic blocking of destructive SQL operations like DROP or TRUNCATE, before any action is committed. The result is an architectural pattern that lets model-driven rigor and LLM agent flexibility coexist in enterprise environments without compromising either.
Borivoj Bogdanović, M. Segedinac, Z. Konjović et al.· SINTEZA· 0 citations
Enterprise large language model (LLM) applications often begin as prototypes whose behavior is carried by prompts and retrieval context. Productization adds requirements for source boundaries, entity routing, answer contracts, and reproducible traces. We present a harness-engineering approach that reconstructs this pattern into a traceable, auditable LLM-agent architecture: deterministic behavior moves into code, manifests, schemas, and validation artifacts around a replaceable composition boundary, while source-backed claims remain the authority for runtime answers. We instantiate it on a public-data slice of five Korean corporate groups (25 listed companies) and evaluate three research questions. (1) The harness preserves its source-grounding, entity-routing, trace, output-hygiene, and recommendation-language contracts across the fixed validation scenarios; a fault-injection control confirms the validators flag deliberately broken contracts. (2) The checks the harness enforces held under model substitution: across three hosted models, they passed on all 270 composition-boundary runs; failures were confined to the model-composed side and were caught and recorded. (3) The code-owned guarantees are load-bearing, not reproducible by prompting alone: holding the model fixed and varying only the enforcement layer, prompt instructions alone let recommendation-language and internal-trace-leakage violations reach the reader, which the harness blocks entirely. A bolt-on external guardrail prevents such violations too but over-refuses, dropping utility to 88/120 where the harness preserves full utility (120/120); in this ablation, only code-owned enforcement preserves both safety and utility. The result is a reusable engineering pattern for turning exploratory prototypes into auditable applications with versioned source, control, and validation artifacts.
This work proposes TraceDev, a multi-agent framework for automated software development grounded in use cases that contain multiple functional points and complex semantics, and demonstrates the effectiveness of TraceDev in repository-level code generation from requirements.
Mingyu Chen, Yakun Zhang, Zihao Xie et al.· 0 citations
Related blog posts
MIT News · Artificial Intelligence· news.mit.eduAug 17, 2026
A USAF cadet and a Lincoln Laboratory researcher found AI chatbots can help nontechnical service members produce viable software applications for their unique problems.
