A Graph-Adapted Semantic Topic Model with Instance-Aware Dual-View Fusion with fusion-type ablation yields counterintuitive findings that support the effectiveness of the adaptive gate strategy over static alternatives.
Abstract
In neural topic modeling, the reliability of a document representation varies from one document to the next: a syntactic view drawn from a dependency graph is informative when the parse is well formed, whereas a semantic view drawn from a pretrained transformer is more dependable when the parse is sparse or fragmented. Existing hybrid models combine the two views with a fixed rule, set once for the whole corpus through concatenation, averaging, or a constant weight, so the locally less reliable view is over-weighted on individual documents. To close these gaps, a Graph-Adapted Semantic Topic Model with Instance-Aware Dual-View Fusion (GASTM) is proposed. Within a variational autoencoder, a GGNN-based dependency encoder and a pretrained transformer semantic encoder are kept jointly trainable, while corpus-wide lexical statistics are carried through a learnable node embedding on the graph branch. A continuous per-document fusion weight is then produced by an instance-aware adaptive gate, allowing the two views to be combined in a controlled manner before reaching the latent inference space. On five benchmarks, GASTM is shown to consistently outperform the baselines on both topic quality and downstream classification. A fusion-type ablation further yields counterintuitive findings that support the effectiveness of the adaptive gate strategy over static alternatives.
Recent advances in neural topic models with pre-trained language models (PLMs) have achieved strong performance by leveraging general-domain pre-training, yet their topic interpretability often degrades on specialized corpora. This limitation primarily stems from the geometry of the embedding space, where domain-specific terms unseen during pre-training collapse into an indistinguishable region, and neither domain-specific re-training, word-level graph enrichment, nor parameter-efficient fine-tuning can restructure this space without inheriting the capacity ceiling of the underlying encoder. Our key insight is that a learnable graph layer operating on token-level PLM embeddings can acquire corpus-specific semantic structure that the frozen encoder lacks, because token-level graphs preserve document-local context that word-level representations discard and joint optimization with the topic objective reshapes embedding geometry directly from target-domain evidence. We instantiate this insight as DARTopic, a domain-agnostic framework that constructs token-level semantic graphs from frozen PLM embeddings and jointly trains a GNN encoder with topic inference. Across three benchmarks spanning general, biomedical, and legal domains, DARTopic consistently outperforms strong baselines in topic coherence and document clus- tering without any encoder fine-tuning, while demonstrating robustness to PLM choice and favorable runtime efficiency over fine-tuning based alternatives.
: Knowledge graphs organize real-world facts as structured triples and have become a fundamental resource for search engines, question answering, recommender systems, and knowledge-enhanced large language models. However, real-world knowledge graphs remain highly incomplete, which limits their downstream reasoning ability. Existing pre-trained language model-based knowledge graph completion methods provide strong textual semantic representations, but they usually model graph structure only as shallow auxiliary features and remain weak in distinguishing structurally similar entities and topology-near negative samples. To address this limitation, this paper proposes a Dual-Level Structural Context Collaborative Framework (DSC 2 F) for knowledge graph completion. At the instance level, the framework introduces Structural Neighborhood Context (SNC) to inject local neighborhood evidence into the language model input and Relation-Aware Attention (RAA) to condition structural aggregation on the current relation. At the batch level, it constructs topology-aware training batches with biased random walk with restart, so that in-batch negatives are locally related to positive samples and impose stronger structural discrimination pressure. Experiments on WN18RR, FB15k-237, and Wikidata5M show that DSC 2 F achieves the best mean reciprocal rank and Hits@1 on all three datasets, consistently outperforming strong embedding-based and pre-trained language model-based baselines. Ablation studies and structural configuration analyses further verify that SNC, RAA, and Batch-Level Structural Context provide complementary benefits. These results demonstrate that collaborative modeling of instance-level and batch-level structural context can effectively enhance structure-aware entity representation and improve fine-grained entity prediction.
Deep sequential encoders such as Transformers are widely used for modeling user behavior in sequential recommendation, yet their effectiveness can be limited when interaction sequences are short and sparse. In such settings, insufficient behavioral evidence makes it difficult to learn reliable representations. Existing contrastive learning methods provide useful auxiliary supervision, but they often rely on random sequence perturbations that may remove informative interactions and typically apply contrastive objectives only to the final encoder layer. Moreover, interaction-based models cannot fully capture global semantic relations among items that rarely or never co-occur. To address these limitations, we propose Semantic Global Graph with Multi-layer Alignment for Recommendation (SeGMA4Rec), which combines item-level semantic enrichment with sequence-level hierarchical regularization. SeGMA4Rec constructs a semantic item graph from pretrained textual embeddings, adaptively refines the retained edge weights, and propagates task-relevant information among related items. The graph-enhanced item representations are then processed by a sequential encoder whose attention scores guide probabilistic view generation toward retaining informative interactions. Three contrastive objectives further align representations across augmented views and encoder layers, providing direct regularization at multiple depths. Experiments on four benchmark datasets show that SeGMA4Rec achieves strong and generally superior performance over competitive baselines, including improvements of up to 9.9% in Recall@10 and 8.1% in NDCG@10 on ML-1M. Additional analyses confirm the complementary contributions of semantic graph propagation, adaptive edge refinement, attention-guided augmentation, and multi-layer contrastive alignment.
This study proposes a teaching evaluation text analysis framework integrating knowledge graph embedding, semantic-enhanced encoding, and joint multi-task learning to address the difficulty of accurately analyzing large-scale unstructured teaching evaluation texts.
Wenbo Li, Heng Wang, Di Yang et al.· International Conference on...· 0 citations
LSR retains a general encoder as a transferable semantic substrate while explicit modules reorganize, augment, or index its representations for a defined operational objective.
Pedro Emílio Amador Salomão· Nexus Science Review· 0 citations
Indic text is syntactically compact, semantically ambiguous, and highly context-dependent, making English translation particularly challenging. Traditional neural machine translation (NMT) methods, even when enhanced with pre-trained language models (PLMs) and linguistic features such as part-of-speech (POS) tags, syntactic dependencies, and semantic role labels (SRLs), struggle to generalise in few-shot settings and often lack consistency across language structures. To address the above limitations, this work proposes a Few-Shot Memory Fusion Transformer (FMFT). The FMFT is a memory-augmented, multi-feature-fusion transformer enhanced with the Retrieval-Augmented Graph (RAG) and Few-Shot learning. FMFT combines multi-granular linguistic signals with a graph-structured semantic memory network, in which nodes represent key tokens (e.g., phrases, named entities) and edges represent purely syntactic, semantic, and contextual relationships. A graph-based dynamic retrieval mechanism that uses Personalised PageRank and a minimum-cost spanning tree enables on-the-fly retrieval of context for each translation instance. This is fused with PLM features via a gated, attention-based multi-feature fusion module that adaptively balances linguistic signals. A multi-head memory attention mechanism incorporates this information into the decoder to enhance semantic clarity and long-range dependency handling. FMFT also learns through non-parametric continual learning with a scalable long-context memory buffer. It adapts to various evolving language inputs without requiring retraining. Evaluation of five Indic-English corpora reveals that FMFT significantly outperforms competitive baselines in both full-resource and few-shot scenarios, achieving improvements in BLEU, ROUGE, METEOR, and Chrf scores. Its handling of rare or ambiguous constructs makes it particularly effective for translating low-resource historical languages. At the same time, its modular graph-based design supports scalability, interpretability, and reuse for domain-specific translation tasks.
S. Chauhan, Shefali Saxena, Shweta V. Jain et al.· IEEE Access· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
