TL;DR
The red herring corrupts the reasoning, and leads a model to rate a problem as easier, not harder, than its clean version, even as it answers that problem wrong more often.
It is shown that reasoning concentrates a model's errors: the probability that two independently drawn wrong answers coincide rises in all ten dataset-scale comparisons, and in nine of nine after restricting both arms to the problems each gets wrong.
Asked to choose between candidates and explain the choice, a language model often rejects a rival by naming a fact its profile lacks: no director, no date of death. That sentence is a claim about the text in front of the model, and it can be tested without any judge. We insert a real corpus sentence stating the named f...
Take three frontier mixture-of-experts models (Alibaba, OpenAI, NVIDIA; 3.6-4.0B active parameters each) and fine-tune them to reason in a low-resource language. On accuracy benchmarks almost nothing happens, and the benchmark itself is noise at this scale: changing only the random seed moves the score by 7.7 points, m...
Evaluating an aligned language model by reading its answers assumes the answers carry the distinction the evaluator cares about. We introduce LLM endognostics, a white-box internal auditing framework designed to extract and causally manipulate latent knowledge within the residual stream. Applied to Minerva-7B-Instruct-...
Large language models (LLMs) often abandon a correct answer, or endorse a user's position, once the user pushes back. This behavior, called sycophancy, is usually reported as a single rate per model, which says little about when it happens or how a user can avoid it. We study the conditions that produce it with 103,939...
This article argues that “AI use” is too blunt a category for judging what happens when a student writes a humanities paper with a large language model. Taking the most intensive case, a paper built with AI from research to final review, it follows the changing relation between student, model, sources, and draft. In re...
We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.
