How users actually use LLM-powered data systems is examined: how they struggle to specify and validate queries, how they iterate extensively to converge on bespoke operators specific to their documents and domains, and what these patterns imply for the systems and benchmarks their community should be building.
Querying LLMs as digital libraries is feasible, but its effectiveness depends on model strength, deployment conditions, dataset structure, and execution strategy, and Galois remains valuable when relational discipline and controlled query execution are required.
Hybrid queries—natural language questions over structured data that require both database capabilities and LLM reasoning—have recently emerged as a prominent research topic. However, existing solutions remain overly dependent on manual workflows, and current benchmarks are limited in scale and diversity. To bridge this gap, we present (1) HyQBench \xspace, a large-scale benchmark with 60\sim 90× more queries than prior work, built on 3× more databases; (2) AutoHyQ \xspace, an automated pipeline that can execute existing methods without manual intervention; (3) multi-dimensional, fine-grained evaluation metrics for comprehensive assessment. Through extensive experiments across multiple hybrid query approaches on diverse LLM backbones, we reveal their strengths and limitations, and identify research opportunities for advancing this emerging field. Our code and data are available at https://github.com/XMUDM/HyQBench.
Bo Li, Chenzhan Wang, Longkang Lin et al.· Proceedings of the 32nd ACM...· 0 citations
In today’s data-driven environment, the ability to explore and manipulate datasets is essential, however, many existing data analysis tools require technical expertise in platforms such as Excel, Python, or Power BI, creating accessibility barriers for non-technical users. This project presents Quiksight, a web-based conversational data analysis system powered by Large Language Models (LLMs) that enables users to interact with datasets using natural language. The objective of this work was to develop an intuitive system that simplifies data analysis by allowing users to upload Excel or CSV files and perform operations such as filtering, sorting, summarization, column modification, missing-value handling, and data export without writing code. The system was implemented using Google Gemini as the LLM, FastAPI for backend processing, and HTML, JavaScript, and Tailwind CSS for the frontend interface.Evaluation showed that Quiksight successfully interpreted and executed a wide range of natural language data manipulation queries with high accuracy. Testing recorded an average query response time of approximately 2 seconds, while dataset uploads up to 30MB remained below 20 seconds under suitable network conditions. Compared with existing platforms such as Julius.ai and PowerDrill.ai, Quiksight demonstrated advantages through its lightweight architecture, simplified interaction model, and focus on everyday users. The project demonstrates the feasibility of LLM-powered conversational interfaces for democratizing data analysis. Future improvements include database integration, support for larger datasets, and advanced analytical capabilities such as statistical modelling.
Wumi Ajayi, Ihuanyachi O. OGBONNA, Daniel I. OGHINAN· Global Journal of Engineerin...· 0 citations