Nishant Raj

Lost in the bf16 Cast: Exporting Ternary Language Models Can Revert Most Low-Learning-Rate Code Changes

Ternary language models such as BitNet b1.58, Falcon-E and BitCPM are fine-tuned with higher-precision latent weights and deployed as ternary codes produced by an export step that, in the labs'documented pipelines, first casts the latents to bf16. We audit those pipelines across three labs. In released checkpoints, fp3...