Filtering search is one of the crucial applications of approximate nearest neighbor search (ANNS). In filtering search, label information is used in conjunction with unstructured vector data. ANNS provides efficient similarity search in large-scale datasets, however, the memory footprint of storing both vector data and...
Kento Tatsuno, Daisuke Miyashita, Kazunari Sumiyoshi et al.· 0 citations
KV-cache reuse in large language model serving is commonly limited to exact prompt prefixes, preventing reuse when the same retrieved document appears after different system prompts, conversation histories, or dynamic metadata. We present KV-Repurpose, an open-source, vLLM-based system for developing and evaluating seg...
Taiga Ikeda, Daisuke Miyashita, Jun Deguchi· 1 citation
The results show that better local reconstruction need not translate into better final-model fidelity, and that the reversal is specific to approximate completion in these cases.
Yasuto Hoshi, Daisuke Miyashita, Jun Deguchi· 0 citations
A combined approach is proposed that fine-tunes the model while taking KV cache concatenation into account and selectively recomputes a subset of the KV caches and demonstrates improved accuracy for long-context inputs.
F. Tachibana, Daisuke Miyashita, Jun Deguchi· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.