A Hybrid Vision Mamba and Transformer Architecture for Offline Recognition of Handwritten Marathi Characters

Abstract

Offline handwritten Marathi character recognition is still kind of hard research problem because there is so much variability within the same class ,and between classes they can look a bit similar ,also the strokes are complex and different people write in their own style. A lot of CNN and Transformer like methods either do not really capture long range relationships well enough, or they end up being too heavy computationally, you know not so efficient. So in this paper we suggest a Hybrid Vision Mamba and Transformer (HVMT), framework for stronger offline handwritten Marathi character recognition. The HVMT idea combines the hierarchical feature extraction power of Vision Mamba, which uses selective state-space modeling, with the contextual representation learning of a smaller Transformer encoder, and inside that encoder we use Multi-Head Self-Attention. Experiments are done on the public MHCD_GIETV2 dataset, where handwritten Marathi characters are collected from writers in different age groups and with diverse writing styles. Before training the images are turned into grayscale, then normalized, resized, and also augmented, to help the model generalize better. The proposed HVMT is compared with CNN, ResNet-50, EfficientNet-B0, ConvNeXt-Tiny, Vision Transformer (ViT-B/16), Swin Transformer-Tiny, and Vision Mamba, all under the same experimental setup. Experimental results show that the proposed framework achieved accuracy 87.11% , precision 87.09% , recall 87.10% and F1-score 87.09% which is better than the compared architectures. At the same time it only uses 26.9 million parameters, 2.6 GFLOPs, and inference time 1.305 ms per image. In other words, the HVMT framework seems to strike a workable tradeoff between recognition precision and compute efficiency. Because of this it is a good fit for things like intelligent document analysis, handwritten document digitization , archival preservation , and several other Indic script recognition tasks and more.