AraBERT-Based Semantic Lexicon Induction for Arabic Academic Text: Query Expansion, Document Clustering, and Classification

Abstract

The scarcity of high-quality semantic lexical resources for Arabic academic text represents a critical bottleneck for natural language processing (NLP) applications, including query expansion, document retrieval, and information organization. This paper presents a framework for inducing semantic neighbor lexicons from the Arabic Research Papers Dataset (ARPD), a publicly available corpus of 2,011 Arabic academic documents spanning seven scientific domains. We exploit type-level representations derived from AraBERT to compute dense, L2-normalized word embeddings and apply GPU-accelerated cosine similarity search to retrieve up to three semantically induced neighbors per vocabulary item. Experiments are conducted on both the raw and preprocessed versions of ARPD. The raw-corpus lexicon covers 168,866 unique terms, and the preprocessed-corpus lexicon covers 159,364, both with 100% three-neighbor coverage and a mean top-1 cosine similarity of 0.91. For document clustering, replacing TF-IDF bag-of-words with AraBERT document embeddings raises the Silhouette coefficient from 0.070 to 0.645 on the raw corpus (+ 820.0%) and to 0.755 on the preprocessed corpus (+ 963.4%), alongside substantial reductions in the Davies–Bouldin index. A pilot expansion experiment under class-based relevance finds no retrieval gain from lexicon expansion, which we report alongside the root-level and expert analyses that explain it. For classification, TF-IDF with only light normalization achieves 99.17% accuracy, exceeding the published benchmark of 99.00% that required heavy preprocessing. Comprehensive comparisons against static embedding baselines (Word2Vec, FastText, GloVe) under both corpus conditions quantify how preprocessing depth interacts with representation type. A comparison against MARBERTv2, AraELECTRA, and CAMeLBERT-MSA under leakage-free, validation-based checkpoint selection shows MARBERTv2 attaining the highest accuracy (99.67% raw, 99.50% preprocessed); we emphasize that differences among the top-performing methods correspond to only a few documents and are not statistically significant, while semantic-neighbor augmentation yields a nominal + 1.82 pp gain for AraBERT on the preprocessed corpus (McNemar $p=0.027$ ).