ForgetBench: Benchmarking Forgetting Dynamics of Long-Term Parametric Memory in Language Models

Towards Evolving Context Parameterization for Large Language Models

Context parameterization enables large language models (LLMs) to internalize contexts into reusable model parameters, avoiding repeated processing across subsequent queries. However, existing methods typically assume static contexts and lack explicit mechanisms for distinguishing validity states under continual updates. To study this real-world scenario, we formalized the Memory Updating with Sequential Evolution (MUSE) task and constructed MUSE-bench to evaluate update incorporation and unaffected-information preservation. The resulting challenge requires preserving the global state while adjusting the contribution of memory evidence. Motivated by this, we proposed PLUME, a training-free method that constructs a global update representation, activates memory evidence to form a local parameter view, and adaptively integrates their predictions during decoding. Comprehensive evaluation on MUSE-bench demonstrated PLUME's effectiveness in sequential evolution settings, yielding relative improvements of 29.9% in average ROUGE-L Recall and 54.9% in LLM-as-a-Judge. Our codes are available at: https://github.com/xiaobingshi-LLM/PLUME.

Towards Understanding Token Evolution in Continual Learning of MLLMs

DynaGraph-LLM: a dynamic ontological memory framework with multi-scale retrieval for mitigating contextual amnesia in large language models

DynaGraph-LLM is introduced, a novel neuro-symbolic architecture that endows LLMs with a dynamic, persistent, and structured memory and implements a Dual-Phase Memory Consolidation process, inspired by hippocampal-neocortical interactions in the human brain, to refine and abstract knowledge over time.

AWARe: Mitigating Catastrophic Forgetting via Activation-Weighted Adaptive REtention

Activation-Weighted Adaptive REtention (AWARe) is a fine-tuning method that mitigates catastrophic forgetting by dynamically controlling parameter updates based on activation patterns, and operates without modifying model architectures, ensuring compatibility with existing inference engines.

Continual Learning Frameworks for Long-Term Knowledge Retention in Evolving Data Streams

Machine learning systems are commonly developed under the assumption of static data distributions, limiting their effectiveness in real-world environments where data evolve continuously. In dynamic domains such as cybersecurity monitoring, financial analytics, and intelligent recommendation systems, models must adapt to new information while preserving previously acquired knowledge. However, sequential learning often leads to catastrophic forgetting, where newly learned information overwrites earlier knowledge representations. Existing continual learning approaches partially address this issue but frequently rely on large memory buffers, explicit task boundaries, or computationally expensive retraining strategies, which limit their scalability in real-world streaming environments. To address this gap, this study proposes a modular continual learning framework that integrates incremental model updates with a memory-based rehearsal mechanism designed to preserve representative samples from previously learned tasks. The framework enables models to adapt to evolving data streams while maintaining knowledge retention with minimal memory overhead. Experiments were conducted using sequential learning benchmarks that simulate realistic data evolution scenarios with varying levels of task similarity and data drift. The results demonstrate that the proposed approach maintains stable predictive performance across tasks while significantly reducing catastrophic forgetting compared with conventional sequential training strategies. In particular, the framework achieves consistent task accuracy with substantially lower forgetting rates and reduced retraining cost, highlighting its effectiveness for long-term adaptive learning. The findings suggest that combining incremental learning with compact rehearsal memory provides a practical solution for building adaptive AI systems capable of sustained learning in evolving data environments.

Synthetic Worlds for Temporal Evaluation and Knowledge Updating in LLMs

This work proposes a synthetic, simulation-driven framework for studying knowledge insertion in LLMs, and introduces {\sc ParallelEvents}, a benchmark of fictional yet realistic future worlds that generates coherent event trajectories for controlled evaluation, avoiding contamination while preserving consistency.