LLM-as-a-Demographic: Whom Sociodemographic Prompting Helps, and Whom It Hurts
It is found that a judge prompted with no demographics is not perspective-neutral: models best reproduce the judgments of White, college-educated annotators and instruction-tuning is identified as a possible source of the asymmetry.