MedQueryIntent: A Large-Scale Benchmark Dataset for Medical Search Intent Classification

Abstract

Understanding medical search intent is critical for improving biomedical information retrieval, particularly for identifying evidence-seeking queries. However, existing resources are limited in scale and lack structured annotation aligned with clinical evidence needs. We present MedQueryIntent, a large-scale benchmark dataset consisting of a human-annotated real-world query set and an LLM-augmented evidence query set. The human-annotated portion, MedQueryIntent-Real, contains 12,000 medical queries collected from Trip and PubMed and is labeled with a hierarchical taxonomy distinguishing evidence-seeking and non-evidence-seeking intent. The annotation follows structured guidelines based on clinical research frameworks (e.g., PICO, PEO, PCC), ensuring consistent and interpretable labeling. To address the limitations of short and ambiguous real-world queries, we further construct MedQueryIntent-Synth, an LLM-augmented evidence query dataset containing 7,633 unique queries, generated by transforming systematic review content into natural query forms via controlled prompting. Together, the two resources form the complete MedQueryIntent benchmark with 19,633 queries. We benchmark multiple models, including biomedical encoders and large language models, demonstrating that MedQueryIntent provides a challenging and realistic testbed. Results show that incorporating MedQueryIntent-Synth improves classification performance and robustness. Our dataset supports research in medical query understanding and downstream applications such as evidence retrieval and clinical decision support. The dataset is publicly available at https://github.com/yingchengsun/MedQueryIntent.