Investigating the Impact of Supervision Format on Reasoning Performance in Large Language Models
TL;DR
Treating supervision format as a first-class hyperparameter for multi-task reasoning SFT in large language models—at least in this benchmark-and-model setting—rather than a mere rendering detail is supported.
Abstract
Supervised fine-tuning is often discussed in terms of data volume and target quality, but the format of the supervision itself can change the reasoning strategy a model expresses in its outputs. We study this effect on a six-category reasoning benchmark spanning symbol transformation, text decryption, bit manipulation, gravitational constant estimation, numeral conversion, and unit conversion (drawn from the NVIDIA Nemotron Model Reasoning Challenge). Using NVIDIA Nemotron-3-Nano-30B-A3B with matched LoRA training settings, we compare three symbol-supervision formats: verbose English rule descriptions, compact family tags, and compact formula notation. We hypothesize that supervision renderings bias token-level reasoning priors, and that these priors transfer across task boundaries in multi-task SFT. In the canonical strict-rescore inventory, the best compact tag and formula checkpoints are statistically equivalent in aggregate within a pre-specified ±4-point margin: K8A-800 reaches 72.3% strict-scored overall accuracy and K8B-700 reaches 71.2% (TOST p = 0.003). Compact tags nevertheless provide a cleaner behavioral profile: an earlier K8A-400 checkpoint reaches 66.4% overall, 98.7% gravity accuracy, and 36.9% bit accuracy without the same contamination signatures. In contrast, verbose English rule descriptions are associated with heuristic parroting, with up to 57% of symbol failures at audited verbose checkpoints collapsing to a single remove-operator template, while formula notation is associated with cross-category contamination: numeric-looking predictions appear more often in text decryption (higher at five of six matched training steps under the canonical seed; matched-step means 15.8 vs. 11.7 numeric predictions per 157 text rows), and gravity failures at a representative K8B formula checkpoint shift toward shortcut stubs and explicit g = 9.8/9.81 fallbacks. We further show that checkpoint selection and strict evaluation auditing materially change branch decisions. Across three training seeds, neither compact format shows a consistent aggregate advantage, while the contamination signatures are partly seed-specific: the gravity-shortcut severity difference persists but is not exclusive to the formula branch, and the numeric–text signature does not reproduce under reseeding. These results support treating supervision format as a first-class hyperparameter for multi-task reasoning SFT in large language models—at least in this benchmark-and-model setting—rather than a mere rendering detail. Because such symbolic and procedural reasoning tasks recur in domains including cybersecurity, mathematics, and code generation, the same formatting choices plausibly shape the policy that any later reinforcement-learning stage would inherit, which we flag as future work.