Small vision-language models may be able to read external evidence yet struggle to obtain it. We introduce Harness Compilation (HC), an offline procedure that adapts the division of work between a frozen small VLM and its external harness. A large teacher uses student execution traces to revise reusable content and con...
Min-Hao Fan, Yin Liu, Jiayu Zhao et al.· 0 citations
Long-context LLM serving is increasingly bottlenecked by KV-cache memory, especially in resource-constrained scenarios. Among existing KV-cache compression strategies, token-wise methods reduce cached states but risk information loss through eviction or condensation, while feature-wise methods reduce per-token KV dimen...
Zi-Han Teng, Jiayu Zhao, Wen-Tao Ren et al.· 0 citations
BitsMoE is proposed, a cost-aware mixed-precision quantization framework built on two complementary techniques that separates expert weights into a shared basis and expert-specific spectral components, defining structural quantization units while exploiting cross-expert redundancy.
Jiayu Zhao, Zi-Han Teng, Min-Hao Fan et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.