From visual detection to traceable progress evidence: A schedule-driven ontology framework for construction monitoring
Abstract
Construction progress monitoring is increasingly combining Computer Vision (CV) and semantic technologies, yet schedules, site photographs, and monitoring outputs often remain disconnected. This limits the interpretability, traceability and reuse of progress-verification results in construction reporting. This research proposes a schedule-driven pipeline that links project planning data and construction-site visual evidence through open-vocabulary object detection, RDF-based Knowledge Graphs, SPARQL queries and controlled LLM-based verbalisation. The project schedule is used to derive and optimise the labels queried by GroundingDINO, while the resulting detections, photographic metadata and schedule information are integrated into a Knowledge Graph to assign progress-related semantic statuses. The LLaMA 3.1 8B model then reformulates the structured SPARQL output into concise technical statements without altering the rule-based classification. The pipeline is tested on the installation of a 100-metre free-standing tower crane at a hotel construction site in Milan. The results show that schedule-driven CV, semantic modelling, and controlled natural-language generation can support transparent, traceable, and semi-automatic documentation of construction progress. The workflow also contributes to SDG 9, SDG 11, and SDG 12 by supporting digital innovation, sustainable construction management, and traceable, resource-aware decision-making.