Open-weight language models from different families exhibit complementary capabilities, motivating their consolidation into a compact student through on-policy distillation (OPD). However, full-vocabulary OPD typically assumes a shared tokenizer, while existing cross-tokenizer methods may discard teacher probability ma...
Hao Wang, Kun Yuan, Wenlin Zhong et al.· arXiv.org· 5 citations· ⚡1
It is found that retaining only the sign of token-level advantages is sufficient to achieve the performance comparable to standard OPD, and a robust TV regulated On-Policy Distillation (TV-OPD) method is proposed, benefiting from the bounded and diminished advantages.
Han Xiao, Yifan Niu, Dong-Yi Liu et al.· 0 citations
Jina-OCR-v1 is presented, an end-to-end document parsing model built to serve on low-budget GPUs that combines the compressed-vision encoder and the 3B mixture-of-experts decoder of DeepSeek-OCR with a FastMTP speculative decoding head that shares a single draft block recursively across K=3 prediction steps.
Omni-macos is presented, which runs its encoder, index and store on the Mac that already holds the files, so no indexed file, no typed query and no vector ever leaves the machine.
Han Xiao· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
