Local Augmentation for Handwritten Text Recognition

Abstract

Data augmentation has proven to help prevent overfitting in deep learning models through various experiments performed on several tasks including Handwritten Text Recognition (HTR). Data augmentation without training is more preferred by researchers than data augmentation using Generative Adversarial Network and diffusion models. However, the indiscriminate application of global matrix transformations to handwritten text images fails to optimally generate samples that conform to the characteristics of human handwriting style. To address this issue, this paper proposes a local augmentation method for HTR. Specifically, we first perform stroke decomposition by skeleton detection. Subsequently, we design a module that incorporates three local augmentation operations. Each stroke undergoes a randomly selected transformation operation. Finally, we collect all the modified strokes to create new handwritten examples. Based on the evaluations in the four repositories, we have achieved good results compared with the best published data augmentation methods recently, and increased both CER and WER by about 5.3% and 5%, respectively. Additionally, We show that local transformations outperform conventional global transformations when applying identical matrix transformation operations to HTR, such as the Affine transformation. These results prove that fine-grained local augmentation helps in increasing robustness and is an effective optimization method for deep learning-based pattern recognition and intelligent perception systems.