TellyTune: Exploring the Limits of LLM-Assisted Tuning for LSM-Based Key-Value Stores

Abstract

Log-structured merge-tree-basedkey-value stores(LSM-KVSs) expose hundreds of configuration knobs to support diverse deployments. Navigating this configuration space remains difficult because good settings depend on workload behavior, hardware limits, resource budgets, and LSM-KVS internal state. Recent LLM-based tuners show promise for generating useful configurations across broad knob spaces. However, existing work primarily demonstrates proposal utility: it does not establish when LLM assistance is practical, where standalone LLM-assisted control breaks down, or what system structure is required to make such assistance safe. This paper presents TellyTune, an LLM-assisted autotuner that identifies where LLM assistance is useful and where explicit system control is required through three modules: 1) context extraction from traces, deployment constraints, and LSM internals to construct tuning context; 2) broad-to-bounded offline exploration for LLM-assisted proposal generation and constrained local refinement; and 3) guarded runtime adaptation with validation, runtime-safe updates, and rollback. We implement TellyTune for RocksDB and evaluate it using db_bench, YCSB, and MixGraph, showing throughput improvements by up to 755% over ADOC, 636% over Endure, and 617% over ELMo-Tune. Our results identify the boundary of LLM-assisted LSM-KVS tuning: LLMs are effective with decision-relevant context, broad offline search, and guarded adaptation to sustained drift, but remain too slow and variable for fine-grained online control.