A Framework for the Systematic Assessment of LLM-Based Ontology Learning in Mobile Networks

Abstract

The increasing adoption of Artificial Intelligence (AI)-native mobile networks has created a growing demand for scalable ontology engineering to support semantic interoperability, knowledge bases, semantically enriched data repositories, and automated reasoning for autonomous network management. Although Ontology Learning (OL) offers a promising approach for automatically constructing ontologies from textual sources, its application to mobile networks has not been systematically evaluated, largely due to the lack of benchmark datasets pairing textual corpora with reference ontologies. To address this gap, this paper presents a publicly available benchmark and a comprehensive assessment of Large Language Model (LLM)-based OL for concept extraction in the mobile-network domain. The benchmark comprises five reference ontologies and their corresponding verbalized textual representations spanning Configuration Management (CM), Performance Management (PM), cybersecurity, and intent management. A reproducible assessment framework is proposed based on cross-ontology analysis, including linear mixed-effects, computational and environmental impact analysis. Experiments with four open-weight LLMs reveal substantial variability in concept-extraction performance across ontologies, with mean F1-scores ranging from 0.72 to 0.29. The linear mixed-effects analysis indicates that most differences among the evaluated LLMs and prompting strategies are not statistically significant, suggesting that increasing model scale alone does not necessarily improve concept extraction performance. At the same time, the results reveal a trade-off between performance and computational sustainability, with smaller models achieving comparable performance while substantially reducing computational and environmental costs. Finally, an additional ablation study reveals a substantial performance gap between ontology-derived verbalizations and independently sourced 3rd Generation Partnership Project (3GPP) Technical Specification (TS) documentation, highlighting the challenges of applying LLMs to concept extraction from dense technical text.