The rapid development of large language models has enabled strong reasoning and decision-making performance across a wide range of domains and complex system tasks such as Design Space Exploration, but their direct deployment on mobile and edge hardware remains constrained.
This work designs an end-to-end integer-only quantization approach to eliminate expensive floating-point computations on Flash compute-in-memory devices, and designs a dictionary-based KV cache compression strategy based on sparse dictionary coding that represents each KV vector as a linear combination of static dictio...
Sebastian Zhao, Minseo Kim, C. Hooper et al.· 1 citation
This study underscores the critical role of such a generalized optimization system in preparing model deployment over resource-constrained heterogeneous hardware in a tactical environment, and proposes a comprehensive hardware (HW) and model-agnostic generalized optimization architecture that integrates these technique...
V. Dasari, Jakob A. Adams, V. Mishra et al.· 0 citations
The three-way intersection of on-device AI inference optimization, retrieval-augmented generation, and Green AI / sustainability has not previously been drawn together into a unified system-design perspective, and this survey undertakes that cross-domain synthesis.
Zhiyuan Cheng, Longying Lai, Yue Liu et al.· 6 citations
EPIC mitigates imbalance via performance-aware expert migration and runtime expert activation, and then improves communication with topology-adaptive transport kernels and fine-grained computation-communication overlap.
Jia-Min Cao, Qingxu Li, Yaozhong Liu et al.· Conference on Applications,...· 2 citations· ⚡1
NoVA is proposed, a technology-architecture co-designed NMP system that overcomes both walls and achieves on average 4.5x higher throughput, 69.8% lower end-to-end latency, and 5x better energy efficiency over a GPU baseline, with only 3.9% area overhead and no loss in memory capacity.
In Jung, Jaeha Min, Joo-Young Kim· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
