HydraML: a hybrid damaged roofs automated machine learning classifier for cultural heritage

Abstract

The protection of cultural heritage is a strategic goal for countries with a rich history, making it both economically and culturally significant. Archaeological sites are fundamental assets and must be continuously monitored and maintained to safeguard all their elements, such as structures, painted frescoes, and statues, which may deteriorate over time. To enhance maintenance and preservation efforts in such contexts, it is essential to keep stakeholders informed about the current conditions of the site. This paper proposes a sustainable Artificial Intelligence solution called HydraML: Hybrid Damaged Roofs Automated Machine Learning Classifier for Cultural Heritage, based on the Fuzzy-Transform. The method combines deep learning and non-deep Automated Machine Learning models for the classification of maintenance issues related to the roofs of archaeological sites. This combination reduces training time and energy consumption compared to the standalone deep approach, aligning with the goals of sustainable artificial intelligence. The performance of HydraML was evaluated on two real-world cultural heritage image datasets developed in this study, representing damage to tiled and flat roofs at the archaeological site of Pompeii, and on a publicly available dataset containing post-disaster images of damaged urban roofs. The results show superior performance compared to existing approaches in the literature, achieving an F1-score of 0.9406 for tiled roofs, 0.9117 for flat roofs in Pompeii, and 0.8686 for the urban scenario, while also achieving lower energy consumption than state-of-the-art deep learning methods.