Beyond Token Alignment: Event Completion for Cross-Tokenizer On-Policy Distillation
ESCD aggregates prefix-related teacher events and supervises the total probability of byte-compatible one-step student completions, avoiding tokenizer-dependent probability splits among individual tokens, and support event entry and event completion as complementary supervision targets for cross-tokenizer knowledge tra...
Jia-Cheng Liu, Jing-Wei Song, Qi-Tuan Zhang et al.
· 0 citations