This work shows that introspection and sampling are complementary but not disjoint on BIRD-Interact-Lite, a Text-to-SQL benchmark with 300 tasks, annotated ambiguities, and an LLM-based user simulator, and proposes a proposed multi-label grammar that substantially narrows it without affecting downstream context.
Results show that SQL verification can be performed with a lightweight learned model while retaining feature-level evidence for inspecting and diagnosing its predictions, and feature attribution shows that the model relies on both semantic grounding and deterministic SQL-structure signals.
N. Shukla, Debasmita Panda, Srutanik Bhaduri et al.· 0 citations
This work presents ParSEval, an interactive demonstration system for configurable, counterexample-driven Text-to-SQL evaluation, which helps developers evaluate iterative model improvements on private benchmarks and bootstrap customized Text-to-SQL test suites without manually crafting diagnostic database instances.
Chun-Yu Chen, Zheng-Jie Miao, Yong Zhang et al.· Proceedings of the VLDB Endo...· 0 citations
This semantic path compilation (SPC) system against direct DDL-to-SQL generation on the ACME insurance benchmark supports an end-to-end systems result, not a causal claim that compilation alone produced the gain, because SPC receives governed semantic artifacts that the DDL baseline does not.
BIRD-History is introduced, a benchmark consisting of 1,393 tasks across 11 databases, designed to evaluate text-to-SQL systems'ability to ground underspecified natural language questions using historical SQL scripts, and a plug-in retriever that extracts five types of external knowledge from historical SQL scripts, th...
Yun-Fan Zhou, Qi-Ming Shi, Yi-Zhou Yang et al.· 0 citations
This work formalizes Text-to-SQL verification as a standalone task: assessing whether a candidate SQL query semantically satisfies a natural language request, without access to ground-truth labels, and proposes and compares two modular verification strategies.
Tarfah Alrashed, Madhup Sukoon, David R Karger et al.· Proceedings of the VLDB Endo...· 0 citations
A comprehensive text-to-SQL benchmarking framework with realistic RBAC constraints is presented, which features an LLM-assisted workflow that augments existing text-to-SQL benchmarks with plausible user roles and access policies, and contains evaluation metrics that identify RBAC-specific failure modes, and disentangle...
Yang Fei, Yangfan Jiang, Yin Yang et al.· arXiv.org· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
