Large language models (LLMs) can generate neural-network modifications, but unrestricted generation is often invalid or harmful. This paper studies a narrower setting: improving a weak target model using a stronger same-family source model from a neural-network database. We propose a source-guided candidate-generation protocol with non-source controls, source-conditioned candidates, and a no-LLM hp_copy ablation under equal evaluation budgets. The protocol reports validity separately from accuracy and selects the best valid candidate only when it improves the target. On CIFAR-10, the strongest source-guided candidate reaches 0.5049 accuracy versus 0.2398 for the best non-source candidate, a +0.2651 advantage, while improving a weak target originally at 0.1254; a five-epoch check preserves the gain at 0.7686 versus 0.4839. On SVHN AlexNet with DeepSeek-Coder-6.7B, source-guided transfer reaches 0.7880 versus 0.2254, a +0.5626 advantage; a fresh repeat reaches 0.8069 versus 0.2509, a +0.5560 advantage. Direct source-recipe copy produces 0.1959 on SVHN AlexNet, matching the original target, while hp_transfer reaches 0.7880, showing that the LLM adapts rather than copies the source recipe. Family-level analysis shows the clearest positive signals for AlexNet, with 6/8 wins across SVHN, Imagenette, and CelebA-Gender, and alt_nn1, with 8/10 wins on CIFAR-10.
: Automatic bug triage is a critical software maintenance task that reduces the high human resource cost of manual developer assignment. In this paper we evaluate various machine learning approaches on Eclipse dataset. We compare classical machine learning approaches like TF IDF + XGBoost, LogReg, NaiveBayes and modern transformer based architectures including: Bert, Nomic, Mpnet, Jina, BGE, E5, CodeLlama, Llama-3.1-8B-Instruct, ChatGPT few-shot prompting. We compare various approaches including using transformer based architectures with frozen weights as embedding with a fine-tuned neural network classification head (MLP). For maximal performance we selected the best performing model E5 (47.63% accuracy) and fully fine-tuned it and achieved 72.49% top-1 accuracy which is the best performing deep learning based approach to the best of our knowledge slightly under performing our best approach: TF IDF+XGBoost 76.47%. Furthermore we also experienced with adding the embedding vectors of some of the aforementioned transformer-based models to the best performing TF IDF+XGBoost models but we experienced slightly worse performance than with only TF IDF+XGBoost. We also experimented with Retrieval Augmented Generation (RAG) and achieved 36.81% top-1 accuracy. The RAG results were primarily influenced by the retrieval rather than the generator.
Márk Lajkó, Balázs Nagy, László Vidács· Proceedings of the 21st Inte...· 0 citations
Test Case Prioritization (TCP) aims to reorder test cases to enable earlier fault detection in Continuous Integration pipelines. Machine learning-based TCP approaches have shown promising results but require substantial historical execution data, creating a cold-start challenge for new or rapidly evolving projects. This paper presents an empirical study of direct transfer learning, where models trained on data-rich source projects are applied to unseen targets without any retraining. A total of 578 experiments are conducted across 17 Defects4J projects (34 within-project baselines and 544 cross-project transfer scenarios) using 270,611 feature vectors. Results show that with appropriate source selection, cross-project transfer can match or surpass within-project baselines. A three-way feature ablation (HIS-only, COV-only, full 13-feature) finds that no feature subset consistently exceeds random ordering (E[APFD]≈0.5) on average, confirming that source selection is the critical variable. Analysis identifies failure-rate alignment and source dataset richness as key predictors of transfer quality, providing source-selection criteria that require no target-side data collection.
Michael Johnson, Khaled Slhoub· International Conference on...· 0 citations
This dataset defines a new basis for reproducible and data-driven AI design, advancing the emerging paradigm of LLM-driven AutoML and architectural generalization across modalities and hardware.
Tolgay Atinc Uzun, Waleed Khalid, Saif U Din et al.· 19 citations
Over the past decade, many test adequacy metrics have been proposed for deep learning that characterize test dataset adequacy from different perspectives, e.g., neuron activation behavior, latent feature coverage, decision-boundary exploration, etc. However, these metrics are typically released as independent research prototypes with substantially different installation and preprocessing requirements, execution workflows, and configuration mechanisms. These complications make them quite difficult to reproduce, compare, and adopt in research work and practical deployment alike. In this paper, we present the engineering details of ADEPT, a framework that integrates representative adequacy techniques, including neuron-coverage-based metrics, surprise adequacy, input distribution coverage, boundary coverage, and source- and model-level mutation score, under a consistent execution workflow. ADEPT provides a template-based metric interface with well-defined extension points for integrating new adequacy metrics. Furthermore, it provides YAML-based configuration management, preprocessing-cache reuse, and structured result reporting, making it easy to use in any research and development workflows. ADEPT is designed for researchers and practitioners who wish to reproduce and apply adequacy metrics without spending days or weeks implementing missing tooling or configuring disparate research prototypes. A demo video is available at https://aub.ie/ADEPT_video.
Yidi Kao, Shawn Burnham, Tommi Rose Fahy et al.· 0 citations
Treating supervision format as a first-class hyperparameter for multi-task reasoning SFT in large language models—at least in this benchmark-and-model setting—rather than a mere rendering detail is supported.
Nhat Thanh Vu, M. Rashid, Fariza Sabrina· Electronics· 0 citations
Large language models (LLMs) are increasingly deployed as critical decision-making components in high-stakes real-world AI systems, rendering LLM reliability a foremost practical concern. In this paper, we focus on enhancing LLM reliability through selective prediction (SP), a strategy that allows an LLM to only predict for inputs where it is likely to be correct (i.e., coverage) and hence reduce the error rate (i.e., risk) on that portion of inputs -- flagging the remaining inputs for future human discretion. In other words, SP improves LLM reliability by balancing the risk-coverage trade-off and enabling seamless human-AI collaboration. To integrate SP into LLMs, we focus on the LLM post-training alignment stage and propose to align LLMs with SP performance metrics, in contrast with existing LLM alignment methods that focus primarily on correctness or calibration metrics. Specifically, we propose a novel alignment framework, Reinforcement Learning for Selection Reward (RLSR), which targets the area under the risk-coverage curve (AURC) -- a popular SP performance metric -- as its alignment objective. RLSR achieves substantially better risk-coverage trade-off compared to multiple alignment baselines on both in-domain and out-of-domain tasks.
Gaoxiang Luo, Yi-Fan Wu, Sinian Zhang et al.· 0 citations