Adaptive Pseudo-Labeling via Word Coherence for Topic Modeling

Abstract

Topic modeling discovers latent semantic structures from document collections, providing interpretable insights applicable across a wide range of domains. However, conventional topic modeling approaches are limited by the absence of labeled data, requiring unsupervised learning for training. To overcome this challenge, we present adaptive pseudo-labeling for topic modeling (APT), a self-supervised learning framework designed to alleviate the need for labeled data. Our framework employs document embeddings derived from pretrained transformers and reconstructs the Bag-of-Words (BoW) representation by directly learning the semantic relationships between documents and words. Simultaneously, APT dynamically generates adaptive pseudo-labels to enhance topic coherence, leveraging word coherence extracted from the BoW representation and semantic relationships among document, word, and topic embeddings. On this basis, we integrate proxy-based deep metric learning into topic modeling to improve semantic coherence and diversity across topics. Accordingly, APT derives latent topics and document representations based on the distances between embeddings in the semantic space. Comprehensive experiments on benchmark datasets demonstrate that our APT framework outperforms conventional topic modeling approaches.