Mohammad Abuzar Ansari

How Reliable Is LLM Confidence in Hindi and Marathi? A Study of Calibration and Answer Stability

Language models are increasingly used to answer exam-style questions in Indian languages, and their confidence scores are often treated as a signal for when an answer can be trusted. We checked how far that trust is justified for two small open-weight models, Qwen2.5-1.5B-Instruct and Qwen2.5-3B-Instruct, using 300 nat...