Ekaterina Vylomova

When Context Sensitivity Isn’t Enough: An Argument Against Likelihood-Based Probes for Evaluating LLM-Simulated Students

Evaluating language models as student simulators currently focuses on whether their dialogue looks realistic, yet it is unclear whether these models actually respond to the quality of teaching the way human students do. However, this has not deterred prior works from using a model’s uncertainty, measured as the log-lik...