Hybrid CNN–Capsule–Transformer Architecture for Indic Handwritten Text Recognition with Cross-Script Evaluation

Abstract

Recognition of Indic handwritten text is a difficult issue with the complex formations of characters, variability of graphemes, ambiguity of strokes, and significant differences between writers. This is particularly problematic in scripts (such as Tamil and Kannada) where the form of the handwritten words and the composition structure often are not regular. Although recent CNN-RNN and CNN-Transformer architecture have achieved encouraging results, they either pay much attention to local visual representation or global context representation and do not consider the structural relationship of handwritten patterns at an appropriate level. This work will offer a solution to this drawback by suggesting a Hybrid CNN 10 Capsule 10 Transformer network with Connectionist Temporal Classification (CTC) to perform end-to-end handwritten word recognition. The proposed framework uses CNN layers to extract local visual features, the capsule module to encode structural and compositional relationships, and the Transformer to learn long-range sequence dependencies to be correctly transcribed. The model is tested on Tamil and Kannada handwritten data to check the effectiveness of cross-scripts. The results of the experiment indicate that the given architecture has a test accuracy of 85.46 percent and a Character Error Rate (CER) of 0.0281 on Tamil and 88.70 percent and a Character Error Rate (CER) of 0.0175 on Kannada. These results indicate that the hybrid framework proposed enhances the performance of cross-script handwritten text recognition as compared to baseline architectures.