Aug 2026· 2026 12th International Conference on Big Data and Information Analytics (BigDIA)· pp. 9-15· 0 citations· 13 references
Abstract
Large language models (LLMs) have rapidly advanced natural-language-to-query (Text-to-Query) capabilities, yet existing public benchmarks remain confined to single database paradigms such as Text-to-SQL or Text-to-KG. They do not capture real-world settings where relational databases, graph databases, document databases, and NoSQL systems coexist. We propose a template-driven method for constructing a multi-database Text-to-Query benchmark targeting heterogeneous data lakes. Our framework introduces a unified semantic entity space anchored by a Global Entity Registry (GER) that maps local identifiers across SQL, Neo4j, MongoDB, and document retrieval systems. From this representation we design 400 query templates and generate approximately 18380 cross-database instances via template-driven entity sampling. Every instance passes execution-level verification and structural consistency validation. The resulting dataset supports single-database queries, cross-database reasoning, and multi-source fusion tasks, offering a standardized resource for evaluating LLMs under heterogeneous data conditions. The benchmark is built on a fully synthetic hierarchical-organization scenario; the template-driven generation method and GER-based entity alignment are domain-agnostic and directly applicable to any multi-model data environment.
This work proposes MDB-Link, a hierarchical schema-linking framework that retrieves question-relevant columns from a global index, aggregates retrieval evidence to shortlist databases, and uses a budget-aware large language model (LLM) for database reranking, table selection, and column grounding.
Beiyu Xu, Zhenyu Wu, Jiao-Yan Chen et al.· 0 citations
This work presents an automated approach for generating query context for relational databases that defines query flows that capture common patterns of data retrieval and analysis questions, systematically generates data models by traversing these flows, and instantiates them with specific values sampled from the datab...
Alekh Jindal, J. Pandey, C. Pavlopoulou et al.· 0 citations
TABERTA is presented, a unified retrieval framework for dataset discovery that adapts a bi-encoder language model to relational data by fine-tuning it on serialized representations of tables by fine-tuning the language encoder to diverse structural patterns using optimized strategies that serialize schema and table con...
Enas Khwaileh, L. Traeger, Albert Gatt et al.· International Conference on...· 0 citations
TQTS-BENCH is introduced, a multi-syntax benchmark for evaluating text-to-query capabilities over TSDBs, and error analysis reveals that this performance gap mainly stems from the heterogeneous query syntaxes across different TSDBs, misinterpretation of time-specific intents, and incorrect schema linking.
Fei Lyu, Zhi-Yi Peng, Jia-Ming Liu et al.· arXiv.org· 0 citations
This paper presents Build2SPARQL, a large-scale benchmark for building KGs generated by a KG-grounded pipeline: SPARQL queries are produced and validated entirely by graph-traversal code, while large language models generate only the natural-language questions, keeping query correctness independent of model behavior.
A conversion pipeline is proposed, driven by coding agents with human-in-the-loop verification, that designs each document schema from expected access patterns and rewrites queries to be MongoDB-native, and builds an access-pattern-based text-to-MQL benchmark (AptMQL-Bench).
Hy Nguyen, Nabi Rezvani, Robin Vujanic· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.