Aug 2026· International Conference on Automated Software Engineering· Vol 33· 0 citations· 33 references
Topic Modeling
TL;DR
A reproducible, human-validated evaluation framework applied to 13 strategies—four architectural families crossed with four reasoning variants crossed with four reasoning variants—across three SLMs spanning 3B–14B parameters, plus targeted ablations.
A large empirical study on using LLMs to generate CI configurations from natural language across services and model families suggests that similarity and validity are distinct objectives for CI generation and motivate schema-aware evaluation and tooling for LLM-based configuration generation.
Two novel contributions are introduced: CodeEval and CodeQual, an open-source execution framework that provides researchers with a ready-to-use evaluation pipeline for evaluating and improving LLMs in software engineering contexts, encompassing both functional correctness assessment and subjective code quality evaluation.
XREPOTEST is introduced, a multilingual repository-level benchmark for unit test generation spanning five underexplored languages: Rust, Go, Julia, PHP, and Ruby, and Invocation Rate is proposed to assess whether generated tests meaningfully exercise the intended functionality.
L. Dung, Dong Cao Van, Nam Le Hai et al.· 0 citations
An automated, multi-dimensional evaluation framework for C# code generation, applying it to four state-of-the-art LLMs: GPT, Gemini, Claude, and Grok is presented and a substantial gap between correctness and quality attributes is revealed.
Seyed Mohammad Mahdi Ghalandarian, Majid Bazargani, Masoumeh Taromirad· 0 citations
A systematic empirical study of multiple strategies for context enrichment and optimization in LLM‐based unit test generation, conducted on seven diverse projects (three open‐source and four proprietary industrial systems), encompassing 261 distinct methods establish this optimized context strategy as a cost‐effective solution for scalable, industrial‐grade automated test generation.
Javier Ferrer, Francisco Chicano· Expert systems· 0 citations
Large language models (LLMs) excel at code-centric and algorithmic tasks, but their software design and modeling capabilities remain limited. Unlike abundant code that enables strong generation performance, the scarcity of high-quality design corpora in the training data restricts their software modeling comprehension. A practical solution is to automatically build large-scale, task-specific datasets for training and evaluating software modeling neural networks. This paper introduces CDGen, a neuro-symbolic pipeline for synthesizing large-scale Text-Class Diagram (TXT-CD) datasets. The pipeline consists of five core phases plus one optional augmentation step: first, a stochastic generator produces CD skeletons with random structures and identifiers; second, an LLM grounds skeletons to business domains by replacing random identifiers with domain concepts; third, CDs are refined via a hybrid LLM and rule-based fixing approach to improve quality; fourth, an LLM generates CD-aligned textual descriptions via few-shot prompting; fifth, alignment validation filters valid description-CD pairs for dataset storage. The optional augmentation step injects controlled noise into texts to mimic real-world description-model inconsistencies. Using this pipeline, we constructed CDSyn10K—a large-scale TXT-CD dataset with 10,000 description-model pairs across 2,000 business domains. Evaluations show CDSyn10K diagrams are highly diverse and comparable to expert-made artifacts. To demonstrate the effectiveness, we fine-tuned Qwen3-0.6B and Qwen3-8B on CDSyn10K to build Qw-OOD-0.6B and Qw-OOD-8B, which surpass the same-size generic LLMs on class diagram modeling tasks.
Xiao He, Jing-Wei Shen, Ru Chen· Proceedings of the ACM/IEEE...· 0 citations
Custom-made molecules are advancing medicine, materials, and agriculture, but producing them is slow and expensive. A new Nature paper highlights RetroChimera, a predictive model that helps accelerate chemical synthesis, helping researchers explore a wide range of molecules. The post Improving synthesis prediction of small molecules at scale with RetroChimera appeared first on Microsoft Research.
MIT News · Artificial Intelligence· news.mit.eduSep 14, 2026
The “HardFlow” algorithm could help generative AI models produce high-quality outputs that obey strict requirements when “pretty close” doesn’t cut it.
AI may appear weightless, but every model depends on physical infrastructure. To understand responsible AI, we need to look beyond algorithms and consider the entire lifecycle of the hardware behind them. The post Responsible AI Must Consider Its Afterlife appeared first on GPT-Lab.
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
