Schema-Conditioned Memory Evidence Ranking for Underspecified Tool Calls: A Controlled Study on Mem2ActBench

Abstract

Large language model (LLM) agents often need historical conversations to recover argument values omitted from current tool requests. This study introduces SCAR–Pair, a two-stage ranker for memory retrieval after tool selection. It separates schema-expanded candidate generation from parameter-level scoring and integrates request, tool, field, and cross-encoder evidence through query-balanced pairwise learning. On a reconstructed Mem2ActBench bank containing 19,661 conversations and 391 evidence-bearing tasks, SCAR–Pair achieved 0.450 Recall@5 and 0.365 nDCG@5 under stratified five-fold evaluation, compared with 0.396 and 0.325 for a matched request-only pairwise ranker. Component-aware intervals and adjusted tests support these task-weighted gains in the ToolACE-dominated mixture. Controlled comparisons support the complete schema-view scoring configuration over request-only scoring on the same candidate pool but do not isolate correct-schema semantics or individual field components as independent contributors. The pairwise advantage over pointwise training is also not significant. Across the tested readers, higher retrieval quality does not yield a significant tool-call accuracy advantage over the strict request-only control. SCAR–Pair therefore improves selected-tool memory evidence ranking within the evaluated setting.