Skip to content
Preprint

Shaping the Document, Not the Model: Optimized Suffixes for Context Following

Oct 2026 · 0 citations · 18 references
Computer Science

TL;DR

This work optimize a document suffix by discrete token search, starting from the answer to a single question and extending the objective to a set of passages, to multi-hop questions that follow from the planted fact, and to later turns of a conversation.

Abstract

When a user supplies a document and wants the answer drawn from it, a language model is expected to follow it even when it contradicts its parametric knowledge. Prior work has focused on making the model rely on context by steering activations, contrasting logits during decoding, or instructing it in plain language. We ask whether the document alone, with no instruction written into it and no access to the model at inference time, is enough. We optimize a document suffix by discrete token search, starting from the answer to a single question and extending the objective to a set of passages, to multi-hop questions that follow from the planted fact, and to later turns of a conversation. On counterfactual passages that contradict parametric knowledge of the model, the suffix flips nearly all items the model resists across three instruction-tuned model families, in most settings outperforms the decoding baseline and the strongest instruction we could write, and matches activation steering. Controls show that the suffix largely acts through the passage rather than injecting the answer, and a single suffix per model transfers to unseen passages. Each extended objective carries the effect to its target, so the objective, not only the document, sets how far the effect reaches.

View source

Similar papers

#artificial intelligence Preprint Oct 2026

From Copying to Composition: How Language Models Develop In-Context Learning

In-context learning (ICL) allows language models to adapt their behavior from instructions and demonstrations in a prompt, but accuracy alone does not reveal how contextual information controls their predictions. We study the acquisition and trainability of ICL with a controlled diagnostic that separates copying a near...

Minglai Yang, Xin-Yu Guo, Ren-Xiong Wang et al. · 0 citations
Preprint Oct 2026

Context or Knowledge: How LLMs Attribute Their Own Answers

Language models can answer a question from a supplied document or from their parametric knowledge. When later asked where the answer came from, what determines their source report? We study this by intervening on the model's key-value (KV) cache after the answer is generated and before the source question is asked. On...

Kirill Afendulev, Alexey Dontsov, Alla Chepurova et al. · 0 citations
#large language models Review Nov 2026

Beyond Truncation: When Can LLMs Safely Omit Generated Text?

Instruction-tuned large language models (LLMs) answer at greater length than a request needs. We ask whether a training-free controller inside the decoding loop can remove that surplus while preserving content, judged against the simplest possible baseline: truncating the model's own answer to the same length. An entro...

Mahmoud Alyosify, Mirna Imbabi, Eman Elkhamisi et al. · 0 citations

The Model Knows When to Stop: Training-Free Early Stopping for Long-Context Reading

Answer-Convergence Stopping (ACS) is introduced, a training-free stopping rule that measures rather than asks, and reveals that by properly utilizing the output signals of frozen models, it can achieve favorable behaviors like adaptive stopping without the need for additional training.

Muath Alyobi, Mohamed Eltahir, Almoayyad Abuljdail et al. · 0 citations

Regime Boundary Alignment for Evidence-Gated Question Answering

Retrieval-augmented language models are expected to answer from the retrieved evidence, but in practice they often keep answering when that evidence is missing. We trace this behavior to the training signal: answer-focused fine-tuning assigns no target to unsupported contexts, so it cannot distinguish a reader that abs...

Zeyan Li, Qi-Rong Guo, Si-Yuan Qiu et al. · 0 citations
#machine learning Preprint Nov 2026

Retrofitting Unbounded Context onto Stock Language Models

A language model that never had to forget would keep every document it has read and every turn it has spoken, and answer from any of it. This paper shows that two stock models can do that on unmodified weights by manipulating their geometry. Rotary attention turns each pair of key dimensions, a plane, at its own rate....

M. Oskin · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.