HIT. A hybrid OCR methodology for document analysis for historical documents
TL;DR
This work provides a validated, privacy-preserving, and locally deployable solution for the high-fidelity transcription of sensitive human rights archives through a hybrid methodology that combines domain-specific fine-tuning for text recognition models with a novel anchoring mechanism to ground VLM generation.
Abstract
The transcription of historical documents from the Chilean dictatorship (1973–1990) is essential for the preservation of memory and the pursuit of justice. However, these archives present significant challenges due to severe physical degradation, noise, and typographic variability, which cause standard Optical Character Recognition (OCR) systems and modern Vision-Language Models (VLMs) to struggle, often resulting in hallucinations or low-fidelity outputs. This thesis addresses this problem by proposing HiT (Here is the Text), a hybrid methodology that combines domain-specific fine-tuning for text recognition models with a novel anchoring mechanism to ground VLM generation. The method consists of two stages. First, we introduce DHiSS and DHiSS+, the first large-scale word-level datasets for this domain, comprising over 110,000 and 185,000 curated images respectively. Second, we present the HiT anchoring pipeline, which leverages a text recognition model, fine-tuned on these datasets to inject high-confidence lexical and geometric cues into a VLM. Experimental results on a representative test set demonstrate that the proposed approach significantly outperforms both local and commercial cloud-based baselines. The best configuration (HiT-DHiSS+) achieves a Word Error Rate of 0.162, representing a 41.9% reduction compared to the unanchored baseline (0.279), while maintaining robustness across a wide range of confidence thresholds. This work provides a validated, privacy-preserving, and locally deployable solution for the high-fidelity transcription of sensitive human rights archives.