Can Small Language Models Review Research Papers? Leniency, Prompt Sensitivity, and Agreement with Human Reviewers on ICLR 2024 and 2025
Four small open language models (Qwen2.5 1.5B, 3B, 7B and Phi-3.5-mini) rate 400 ICLR 2024 and 2025 submissions from their title and abstract, using four prompts that ask the same question in different words. The study measures how lenient the models are, how much the wording changes their verdict, how close they come...