Ze-Shen Su

KVMem: Virtualizing Million-Token Agent Workspaces on a Consumer GPU

Modern LLM agents operate in persistent workspaces whose accumulated history can exceed both GPU KV capacity and the model's native context window. Existing systems typically compact older context into summaries or retrieve it later as text, either losing fine-grained execution evidence or repeatedly prefilling content...