Narjes Sharafi

Is Memorization Context-Sensitive? Prefix-Based Extraction Beyond Isolated Prefixes

Large language models (LLMs) can expose memorized training sequences under prefix-based extraction: given a prefix from a training example, the model may assign high probability to the original continuation. In deployed systems, however, prefixes are rarely evaluated in isolation. They often appear together with instru...