Improving Heart-Focused Medical Question Answering in LLMs via Variance-Aware Rubric Rewards with GRPO

Abstract

Large language models can support medical question answering, but privacy, cost, and reliability still limit their clinical use in focused domains. Can rubric-based reinforcement learning bring a locally deployable model close to much larger systems? We study Group Relative Policy Optimization (GRPO) for heart-focused medical question answering with rubric supervision from RaR-Medicine. Rather than rely on binary rubric aggregation or a single overall score, our variance-aware reward framework builds continuous rewards from criterion-level outcomes. It preserves partial credit and reduces reward sparsity. We adapt a Qwen3-14B policy with low-rank adapters and quantized weights. On a held-out heart-related HealthBench subset, the best GRPO variant improves accuracy from 0.362 to 0.502 and F1 from 0.532 to 0.668 over the base model, close to GPT-OSS-120B. An independent clinician rated most responses clinically acceptable (61%) and patient ready (63%), both above the automated accuracy; fewer than 2% were unsafe. Taken together, the results support rubric-based reinforcement learning as a practical route for focused medical language models.