Industry applications often demand low-latency classification, yet current large language model (LLM) approaches remain poorly suited for latency-critical applications. Existing prompting and constrained decoding produce verbose, multi-token outputs that require expensive token-by-token generation, while encoder-based...
Sheng-Hong Dai, S. Pentyala, Ying-Chi Liu et al.· 1 citation
Edge SoCs combine CPUs, GPUs, and NPUs, yet runtimes for multimodal large language models (MLLMs) keep generation on one engine and leave others idle. Streaming workloads allow ingestion and answering to overlap, but backend-local KV state binds queries to the engine that built their history. Stage performance and co-r...
Yi-Long Li, Cheng-Po Yan, Yi-Jing Zeng et al.· 0 citations
This work coordinates heterogeneous accelerators to execute stateful VLM inference concurrently and beats the strongest single-engine baselines on Apple M4 Max, Rockchip RK3588, and Qualcomm QCS6490, even when they reuse context.
Yi-Long Li, Cheng-Po Yan, Yi-Jing Zeng et al.· 0 citations
Koa-action is a framework for low-latency atomic actions -- fast, single-step decisions such as classification, semantic endpointing, Boolean checks, and scoring -- formulated as constrained generation with single-token outputs by introducing atomic label tokens and applying supervised fine-tuning.
Sheng-Hong Dai, S. Pentyala, Ying-Chi Liu et al.· 0 citations
Obtaining the optimal action-value function in Markov decision processes is computationally intensive in large state--action spaces. In this study, we present statistically rigorous convergence results for a robust reinforcement learning algorithm warm-started by a transformer-based action-value function prediction, wh...
This paper proposes a deep learning model formally called QEmbed, built upon the Masked Autoencoder for Distribution Estimation (MADE) auto-regressive framework to learn joint data distributions for selectivity estimation, and designs a hybrid encoding scheme that combines one-hot and embedding encodings.
Pooja Rajput, Suman Banerjee· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.