Transformer Framework with Dynamic Learning Rate Optimisation for Offline Devanagari Character Recognition

Abstract

Purpose: In this study, we explored the effect of multi-scale hybrid deep learning architectures and adaptive optimisation on offline Devanagari handwritten character recognition (HWCR) performance. CNNs are local networks that have been shown to excel at extracting local features but are unable to process the context of complex character configurations. Design/Methodology/Approach: This paper introduces a unified hybrid multi-attention deep learning framework which systematically combines the complementary merits of CNN layers, the Vision Transformer (ViT), and the Swin Transformer. In conjunction with a sequence-modelling layer based on Bidirectional Long Short-Term Memory (BiLSTM) and a Connectionist Temporal Classification (CTC)- based decoding layer, this visual extraction pipeline eliminates the need for explicit character segmentation. The handwritten Devanagari character dataset from Kaggle was used, but images were resized to 64 x 256 to define the features of input images and define the evaluation benchmarks. To prevent premature changes in the parameters and to pass through plateaus more gracefully, we adopted a dynamic learning rate optimisation scheme based on linear warm-up, cosine annealing, and validation loss. Research Limitation: Misclassification issues remain for localised areas with highly stylised writing, overlapping strokes, or partially faded handwriting lines. Findings: The baseline CNN + BiLSTM architecture achieves an accuracy of 93.18%. Incorporating global attention via the Vision Transformer improves accuracy to 95.02%, and adding hierarchical localised window attention through the Swin Transformer increases performance to 96.41%. Introducing the adaptive learning rate strategy yields the final proposed model pipeline, which outperforms all standalone and baseline configurations, achieving a peak classification accuracy of 97.86%, a weighted precision of 0.9829, and a weighted F1-score of 0.9783. Practical Implication: For the practical implementation and evaluation of the end-to-end framework, we have used the PyTorch deep learning library in a GPU-enabled computing environment. Social Implication: This research will contribute to improving automated document indexing, archival digitisation, and language accessibility services for widely used scripts in Indian languages. Originality/Value: A unified multi-attention framework was developed that explicitly optimises structural handwritten character script recognition through an adaptive multi-phase learning-rate schedule, incorporating global and hierarchical local transformer operations (ViT and Swin Transformer), respectively.