Machine learning (ML) increasingly powers Internet of Things (IoT) applications at the edge. Yet producing a deployable edge ML artifact for a specific scenario requires navigating a huge search space spanning data representation, model design, training on domain-specific data, and runtime customization. This workflow...
Gen Wang, Kai-Wei Liu, Lie-Kang Zeng et al.· 0 citations
This work introduces CacheRepair, a lightweight network that learns the difference between independently computed KV caches and those produced by processing the chunks together, and shows that repair reduces KV errors both near chunk boundaries and throughout chunk interiors.
Gen Wang, Wang-Song Yin, Yeerzhati Abudunuer et al.· 0 citations
HA-NPU is presented, the first system to enable efficient hybrid attention LLM inference on edge NPUs without modifying the underlying algorithms, and enhances execution efficiency by reorganizing the dataflow of the LA components across three levels.