Large language models for evidence-based planning: Evaluating an SLR-RAG framework for knowledge synthesis of urban vacant land

Data Foundations of Long-Context Language Models: A Survey

As the context window of Large Language Models (LLMs) continues to expand, the data required to effectively train and evaluate these capabilities remains underexplored. With existing research primarily focuses on architectural optimization, there is a need for a systematic, data-centric review. This survey bridges this gap by investigating the data foundations of Long-Context Language Models (LCMs). We begin by examining current data strategies alongside their strengths and limitations, mapping the required data to desired model capabilities. Building on this, we explore how targeted training data designs drive core, often interconnected skills such as retrieval, reasoning, and aggregation. Furthermore, we analyze the evaluation landscape, illustrating how selecting appropriate benchmarks is crucial for probing capability boundaries and guiding effective model selection. Finally, we synthesize actionable guidelines for data construction and outline critical future directions to propel the advancement of long-context language models, including quantifying data quality, establishing scaling laws for length distributions, and developing dynamic evaluation frameworks.

Retrieval-Augmented Large Language Model for Institutional Knowledge Management and Decision Assistance in Public Organizations

The increasing volume and complexity of institutional documents in public organizations create challenges in accessing reliable knowledge for administrative processes and evidence-based decision-making. Conventional knowledge management systems often rely on keyword-based retrieval, while standalone Large Language Models (LLMs) may generate inaccurate responses when processing domain-specific institutional information. This study proposes a domain-specific Retrieval-Augmented Generation (RAG) framework to enhance institutional knowledge management and AI-assisted decision support in public-sector organizations. The framework was developed using a Design Science Research approach with Universitas Malikussaleh as a case study. The proposed architecture integrates institutional knowledge base construction, semantic retrieval, grounded language generation, and source attribution mechanisms. A knowledge base comprising 416 official institutional documents was developed through document preprocessing, semantic chunking, embedding generation, and vector database indexing. The framework was evaluated using 200 institutional queries based on retrieval performance, response quality, explainability, and system efficiency metrics. The results demonstrate effective retrieval capability, achieving Precision@5 of 0.884, Recall@5 of 0.921, and Mean Reciprocal Rank of 0.895. Generated responses achieved 94.6% factual accuracy, 91.8% contextual relevance, and 96.5% source attribution accuracy, while the hallucination rate was reduced to 3.2%. Furthermore, the framework achieved an average response latency of 1.18 seconds, indicating practical feasibility for institutional applications. These findings demonstrate that integrating semantic retrieval with grounded LLM generation can improve knowledge accessibility, transparency, and reliability for AI-assisted decision support in public organizations. The proposed framework provides a practical foundation for trustworthy institutional knowledge services and supports more efficient, explainable, and evidence-based administrative decision-making across diverse institutional contexts

Leveraging Large Language Models for Automated Assessment and Mapping in Participatory Urban Planning

Abstract. This research introduces an innovative platform designed to enhance citizen engagement in urban planning and management by integrating emerging technologies such as Artificial Intelligence (AI), Large Language Models (LLMs), and chatbots. Traditional Public Participation Geographic Information Systems (PPGIS) often face challenges in effectively capturing and analyzing citizen input. This platform addresses these limitations by enabling users to articulate urban issues or ideas in natural language, which are then processed through AI-driven Natural Language Processing (NLP) techniques to identify key elements such as location, issue type, and intensity. Furthermore, the platform facilitates interactive dialogues, allowing citizens to inquire about perspectives from other community members, thereby fostering a dynamic exchange of views. In the absence of an initial user base, a dataset comprising 2,000 tweets related to Montreal's public transportation was curated. An LLM was fine-tuned using this data, equipping the model to respond to queries concerning Montreal's public transportation system. The findings demonstrate the feasibility of leveraging AI and LLMs to create a responsive and interactive platform that not only streamlines data collection but also enriches the participatory planning process. This approach has the potential to transform urban governance by making it more inclusive and data driven.

Evaluating RAG approaches for Question Answering

RAFE-XAI: A Retrieval-Augmented Feature Engineering and Explainable NLP Framework for Urban Infrastructure Risk Classification

Urban infrastructure systems increasingly depend on textual reports generated by citizens, inspection teams, maintenance units, emergency platforms, and smart city services. Accurate identification of critical risks in these reports is essential for enhancing urban resilience and enabling timely decision-making. Nevertheless, urban infrastructure risk classification is challenging due to the brevity, noise, domain specificity, and context dependence of these reports. This study introduces RAFE-XAI, a retrieval-augmented feature engineering and explainable natural language processing framework for urban infrastructure risk classification. The term retrieval-augmented is used here in a classification-oriented sense: retrieved reports are used to construct additional features and evidence, not to generate output text as in Retrieval-Augmented Generation systems. The proposed framework incorporates semantic sentence embeddings, retrieval-based evidence, neighborhood-derived label distributions, domain-specific risk indicators, infrastructure asset cues, location indicators, and evidence-based explainability. The framework does not construct an explicit graph, adjacency matrix, graph neural network, or message-passing mechanism. Instead, retrieval is used to derive neighbor label-distribution features, which are combined with semantic embeddings and interpretable keyword, asset, and location indicators. To assess the effectiveness of this approach, UIR-Text, a semi-synthetic urban infrastructure risk dataset with scenario-level group splitting to mitigate data leakage, was constructed. Experimental results on UIR-Text show that fine-tuned DistilBERT achieves the strongest predictive performance, with Macro-F1 scores of 0.8278 for category classification, 0.9120 for binary critical-risk detection, and 0.3379 for four-level severity classification. Among the explainable feature-engineering models, RAFE-XAI with Random Forest achieves the strongest category classification performance, with Accuracy 0.8400, Macro-F1 0.8043, Weighted-F1 0.8444, and MCC 0.8062. These results suggest that fine-tuned transformers provide the highest predictive performance on this benchmark, while RAFE-XAI offers a transparent retrieval-augmented alternative that exposes retrieved evidence, neighbor label distributions, and domain cues. Four-level severity classification remains challenging, even with fine-tuned DistilBERT, indicating the need for richer impact-aware variables.

CoAL-RAG: A Complexity-Aware Legal Retrieval-Augmented Generation Method

Legal consultation questions exhibit multi-level complexity. A single retrieval strategy often leads to over-reasoning for simple questions and poor interpretability for complex ones, making it difficult to meet the requirements for both answer quality and efficiency in high-risk scenarios. To address this issue, this paper proposes CoAL-RAG, a complexity-aware legal retrieval-augmented generation method, which constructs a multi-dimensional evaluation mechanism based on ``question essence''and ``retrieval consistency''to enable adaptive routing of retrieval strategies. First, the reasoning demand is quantified according to the logical structure of the question. Then, the discrepancy between semantic retrieval and keyword retrieval is utilized to indirectly reflect problem complexity, thereby selecting the most appropriate retrieval strategy and dynamically filtering contextual information. Experimental results demonstrate that the proposed method significantly outperforms baseline models not only on Chinese legal benchmarks (SocialLawQA, LawBench) but also demonstrates strong cross-jurisdictional generalization on English datasets (LexGLUE, CaseHold). Specifically, on Chinese datasets, the BLEU score improves by 42.5\% and ROUGE-L reaches 3.6 times that of knowledge graph-based methods. On English benchmarks, CoAL-RAG maintains highly competitive accuracy, achieving an optimal balance between generation quality, deep logical reasoning, and system efficiency across different legal systems.