A Comprehensive Taxonomy of Temporal Dimensions in Natural Language Queries
TL;DR
A formal and comprehensive taxonomy for classifying natural language queries along four dimensions, designed as a foundation for diagnostic evaluation of QA and IR systems by providing a systematic categorization of the temporal properties of queries.
Abstract
Effectively understanding and modeling the temporal aspects of user queries is crucial for Information Retrieval (IR) and Question Answering (QA), particularly in contexts that demand freshness, historical accuracy, or temporal reasoning. In this paper, we present a formal and comprehensive taxonomy for classifying natural language queries along four dimensions: (i) temporal understanding, (ii) reasoning type, (iii) time sensitivity, and (iv) trendiness. This framework captures a wide range of temporal intents, from static factual questions to dynamic, event-driven queries. The taxonomy is designed as a foundation for diagnostic evaluation of QA and IR systems by providing a systematic categorization of the temporal properties of queries. This structure enables practitioners to identify, isolate, and analyze system behaviors across diverse temporal scenarios, helping uncover specific failure modes related to temporal reasoning, sensitivity, data freshness, and relevance. We apply the taxonomy to classify queries from four public datasets: MS MARCO, FreshQA, RealtimeQA, and SituatedQA, revealing systematic gaps and underrepresented time-sensitive categories. As a diagnostic case study, we stratify the accuracy of three LLMs by temporal dimension, showing that the taxonomy exposes failure patterns that the aggregate metrics conceal.