Embodied and edge systems increasingly rely on vision-language models (VLMs) to support heterogeneous skills, yet deploying a separate task adapter for each skill while retaining a high-precision backbone creates substantial memory and management costs. We present UniCompress, a deployment-oriented framework that conve...
Ming-Kang Liu, P. Jiao, Wei-Ming Meng et al.· 0 citations
PARC is presented, which combines parallelism-aligned structured pruning with lifetime-aware KV-cache reorganization, which uses first-order Taylor importance and grouped-query-attention-aware selection to make attention dimensions divisible by the target core count while preserving the query-to-key/value ratio and hea...
P. Jiao, Ming-Kang Liu, Wei-Ming Meng et al.· 0 citations
Speculative decoding accelerates large language model inference by employing a compact draft model to propose candidate tokens that are verified in parallel by a target model. Conventional training relies on soft-label distillation, which forces the draft model to mimic the target's full output distribution, including...
Ming-Kang Liu, Wei-Ming Meng, P. Jiao et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.