Hao Fu

The Value of Perfect Information about the Query in Budget-Constrained LLM Memory

Under token budgets, memory systems for Large Language Models (LLMs) select historical information before receiving a query (query-agnostic selection) or after receiving it (query-aware selection). To isolate the value of knowing the query, we idealize the continuous text of each history as a sequence of fixed chunks a...