A Modular Information Extraction and LLM-Based Automated Reporting Framework for Multilingual Disaster Analysis

Abstract

This study presents a modular information extraction architecture built on a multimodal disaster dataset containing original image and text data to interpret unstructured content shared on digital platforms after disasters, including Instagram, Twitter, and Facebook. The study aims to extract critical information with high accuracy from Turkish and English raw texts and transform it into structured reports in English. The proposed system consists of two complementary layers. In the first layer, location detection is performed using Dictionary-Based Named Entity Recognition (NER), with the Turkey Province-District List dataset serving as a validation dictionary to isolate high-precision data, while disaster magnitude detection is carried out using context-aware Regular Expressions (Regex). This deterministic approach achieved an F1-score of 95.66% for location detection and 99.40% for disaster magnitude detection. In the second layer, Qwen-2.5 and LLaMA-3 models fine-tuned with the LoRA method are used to interpret the broader context of the text and generate reports from multilingual inputs. In the developed system, standardized reports containing disaster type, location, and magnitude information are produced by combining deterministic outputs with the semantic inference capabilities of LLMs. Comparative results show that LLaMA-3 achieved the highest performance, with an F1-score of 93.80% and an accuracy of 92.50%, while Qwen-2.5 achieved an F1-score of 91.03% and an accuracy of 90.0%. Overall, the study evaluates the effectiveness of the proposed approach for structuring disaster-related data using Qwen and LLaMA models.