LMEnt: A Suite for Analyzing Knowledge in Language Models from Pretraining Data to Representations

Co-LMLM: Continuous-Query Limited Memory Language Models

This work proposes continuous-query LMLM (CO-LMLM), where the KB pairs continuous keys with textual knowledge values, a significant departure from prior reliance on relational KB and queries, which outperforms prior LMLMs and vanilla LLMs in both perplexity and factual precision.

Knowledgeless Language Models: Suppressing Parametric Recall for Evidence-Grounded Language Modeling

The results demonstrate that suppressing entity-linked supervision during pretraining induces a shift in epistemic behavior: KLLMs rely less on parametric knowledge and more on external evidence, leading to improved reliability under realistic conditions, suggesting that pretraining-time control over knowledge acquisition can complement retrieval-augmented and tool-based systems by providing a more evidence-sensitive base model.

Instructing the Learning of Language Model with the Token Interpretation to Improve Language Understanding

Augmenting Datasets for Fine-Tuning Large Language Models Using Semantic Variations

This study explores a semantic variation methodology to augment training data by generating question-answer pairs with explicit control over semantic similarity, and shows that semantically controlled augmentation improves domain-specific knowledge acquisition while preserving consistency.

Integrating Heterogeneous Knowledge for Enhanced Recommendation with Large Language Models

The integration of Large Language Models into recommender systems has introduced a new paradigm in which models leverage their pre-trained knowledge to generate recommendations. A prevailing assumption is that an LLM’s inherent knowledge is sufficient to support high-quality recommendations across diverse domains. This paper challenges that assumption, positing that in specialized domains, recommendation efficacy is limited by the textual nature of an LLM’s knowledge. To address this limitation, we propose REKALM, a comprehensive integration framework for enhancing LLM-based recommenders through knowledge integration. Central to our approach is the extension of established text-conversion techniques to non-traditional data modalities. We utilize knowledge lexicalization, a process that translates heterogeneous data sources into a unified natural language format. This lexicalized corpus is then used in a knowledge-aware instruction-tuning pipeline to explicitly align the LLM’s internal representations with four distinct types of domain-specific information. We conduct experiments across four distinct domains to validate our framework. Our findings provide consistent empirical evidence that while an LLM’s inherent knowledge may suffice for universally familiar domains like movies, recommendation quality in more specialized areas is significantly improved through knowledge integration. The proposed approach demonstrates that augmenting LLMs with lexicalized, domain-specific knowledge is an effective system-level strategy for advancing the next generation of recommender systems.

Inventive Meta-Learning Strategies for Cross-Domain Natural Language Understanding in Distributed Knowledge Bases

Natural Language Understanding (NLU) addresses cross-domain challenges in understanding language due to inconsistent data distributions, uncertain concept semantics, and inconsistent language use specific to individual domains across different geographical locations. The majority of classical deep learning systems perform poorly because it cannot generalize, leading to cross-domain underperformance in practical settings. Therefore, this research develops a new class of meta-learning that provides a systematic means to advance cross-domain adaptability for NLU in the context of distributed knowledge systems. More specifically, this approach combines federated learning methods, model-agnostic meta-learning (MAML), and domain-adaptive transformers for the first time to enable rapid extraction of domain-invariant representations while concurrently maintaining local contextual semantics across reasonably distributed datasets. The author conducted benchmark assessments on the most recent multi-domain evaluations and cross-lingual and domain-shifted data to simulate distributed knowledge systems. The custom model outperformed all baselines, transformers, and fine-tuning models by an average of 12.8% cross-domain accuracy, 15.3% lower generalization error, and 18.6% faster convergence rate. In the scenario, the new model decreased to less than 90% and maintained over 90% of its performance after a 40% reduction in training samples. Meta-learning enables superior transfer efficiency and model adaptation to new, previously unseen domains with minimal training. The new meta-learning frameworks successfully addressed the required performance and adaptability for advanced intelligent information systems for cross-domain NLU in distributed knowledge bases.