A Formal Trustworthiness Construct for Large Language Model-Based Test Generation: A Multidimensional Index Empirically Evaluated Through a Multi-Agent Study

Abstract

Software code testing remains a critically important but labour-intensive process in software quality assurance. Existing research evaluates large language model (LLM)-based unit test generation using various quality metrics, such as correctness, coverage, mutation score, and test code smells. However, these single metrics do not reflect the trustworthiness of the unit test generation process. Therefore, this research formalises the trustworthiness of LLM-based unit test generation as a multidimensional index comprising reliability, hallucination resistance, maintainability, functional completeness, and human-reference alignment. In this research, we investigate the effect of prompt engineering strategies on the trustworthiness of LLM-generated unit tests and compare them with human-written tests for the same focal methods. Each dimension is fed by a distinct artefact-level measurement and grounded in dependability theory and ISO/IEC 25010:2023. A centralised multi-agent system generates, builds, repairs, and measures the tests, so that all inputs are collected automatically. The index is evaluated on real-world C# focal methods across 18 model × prompt configurations and a paired human-written baseline. The human baseline achieves the highest T-UTG value (0.904), and the best configuration, Combined × Gemini, achieves 0.788. Entropy weighting identifies maintainability and hallucination resistance as the most discriminating dimensions, and a rank-acceptability analysis over the whole weight simplex confirms that this ordering does not depend on the chosen weighting scheme.