SSRFT shows substantially greater robustness to prefilling attacks and better generalization to unseen jailbreak domains, while reducing over-refusal on benign queries and preserving the model's general capabilities, establishing safe-role internalization as an effective alternative to refusal-centric safety alignment.
Jin-Hao Pang, Jitai Hao, Qiang Huang et al.· 0 citations
HEAR is established as a reusable coordination substrate for efficient agentic LLM serving by separating protocol semantics from optimization policies, and Mooncake shows that workflow intent and live engine state provide complementary benefits across load regimes.
Jia-Qi-Fu-Quan Zhao, Hao-Dong Chen, Jitai Hao et al.· 0 citations
Long-context LLM agents accumulate interaction histories that strain KV-cache memory and attention computation. Although sparse attention reduces these costs, heterogeneous cache representations and workflows hinder integration with existing inference engines, while prior sparse-serving abstractions support only specif...
ResidualKV is proposed, which factorizes the KV cache into a sparse set of globally retrieved references and compact, quantized residual codes for the remaining tokens, and shows that global cross-token redundancy supports accurate, memory-efficient, and computation-efficient long-context inference.
Jitai Hao, Qiang Huang, Yaowei Wang et al.· 10 citations· ⚡1
ShallowStream is a novel framework that leverages the shallow layers of an MLLM to simultaneously perform frame encoding and retrieval index building and achieves performance on par with the strongest existing streaming methods, while reducing per-frame prefill latency and 10-second end-to-end latency by up to 52.1x an...
Jitai Hao, Ke-Shuai Yang, Ding-Kun Yan et al.· 2 citations
Low-Rank Clone (LRC), an efficient pre-training method that constructs SLMs aspiring to behavioral equivalence with strong teacher models, and maximizes knowledge transfer while removing the need for explicit alignment modules.
Jitai Hao, Qiang Huang, Hao Liu et al.· Neural Information Processin...· 17 citations· ⚡2
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.