Large Language Models (LLMs) have brought transformative advancements to natural language processing. However, their massive computational and memory requirements—stemming from tens of billions of parameters—make deployment on resource-constrained edge devices such as mobile phones and embedded systems highly challenging. Consequently, extensive research efforts have focused on enabling efficient on-device LLM inference. This paper provides an in-depth survey of two major inference acceleration techniques for on-device LLMs: KV cache optimization and speculative decoding. We present the core principles, representative studies, and performance characteristics of each approach. Furthermore, we discuss open challenges and future research directions, aiming to contribute to the advancement of efficient on-device LLM systems.
Mingyu Jo, Sangoh Park· International Conference on...· 0 citations
Serverless computing, particularly Function-as-aService (FaaS), has emerged as a dominant cloud computing paradigm offering automatic scaling, pay-per-use pricing, and simplified deployment. However, the stateless and ephemeral nature of serverless functions introduces significant performance challenges, most notably cold-start latency and costly remote data access. Caching has been widely recognized as a key technique to mitigate these issues, yet the unique constraints of serverless environments-such as ephemeral containers, limited local state, and multi-tenant resource sharing-demand fundamentally new caching designs. In this paper, we present a concise survey of five representative caching systems for serverless platforms published between 2021 and 2025. We categorize them into three classes: container-level caching for cold-start mitigation, datalevel caching for reducing storage access latency, and cache coherence mechanisms for distributed consistency. We compare these systems along key dimensions including caching granularity, eviction strategy, transparency, and performance gains. Finally, we identify open challenges and promising directions for future research.
Donghyeon Kim, Sangoh Park· International Conference on...· 0 citations