A mixed-method, between-subjects user study comparing SQL-LLM, a GPT-4o-backed NLIDB, with Snowflake, a traditional SQL analytics platform finds that a verification burden survives the switch to natural language, so an NLIDB that hides the generated SQL would remove the very step users relied on to trust the answer.
The study empirically demonstrates comparative behaviours of the current LLMs, and suggests the use of execution-aware evaluation in the studies and deployment of Text-to-SQL.
B. N, T. L. Manasa· International journal of com...· 0 citations
This paper explores the practical implementation of a Chatbot designed to connect natural language communication with database interactions by generating Structured Query Language (SQL) queries through the utilization of OpenAPI, with potential implications for improving human-computer interactions within database syst...
R. Santosa, Hartantya Ainiyatuts Tsaniyah, Yoga Ari Tofan et al.· IPTEK: The Journal for Techn...· 0 citations
How users actually use LLM-powered data systems is examined: how they struggle to specify and validate queries, how they iterate extensively to converge on bespoke operators specific to their documents and domains, and what these patterns imply for the systems and benchmarks their community should be building.
Shreya Shankar, Aditya G. Parameswaran, U. Berkeley· 0 citations
It is suggested that structured prompt engineering provides a practical alternative to model fine-tuning for locally deployed LLMs, offering an effective balance between SQL generation accuracy, computational efficiency, and data privacy.
Nurjayanti Nurjayanti, A. Adiwijaya, A. Romadhony et al.· Jurnal RESTI (Rekayasa Siste...· 0 citations
BIRD-History is introduced, a benchmark consisting of 1,393 tasks across 11 databases, designed to evaluate text-to-SQL systems'ability to ground underspecified natural language questions using historical SQL scripts, and a plug-in retriever that extracts five types of external knowledge from historical SQL scripts, th...
Yun-Fan Zhou, Qi-Ming Shi, Yi-Zhou Yang et al.· 0 citations
This work shows that introspection and sampling are complementary but not disjoint on BIRD-Interact-Lite, a Text-to-SQL benchmark with 300 tasks, annotated ambiguities, and an LLM-based user simulator, and proposes a proposed multi-label grammar that substantially narrows it without affecting downstream context.
Leonhard Liu, Patrick K. Erdelt, Two· 1 citation
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
