When Consistency Lies: Evaluating Prompt Paraphrasing for Black-Box Detection of Incorrect LLM Answers
Can a model’s answers to repeated factual questions help identify when it is wrong? A wrong answer may be repeated consistently, so this study compared two black-box signals: answers to identical prompts and answers to meaning-preserving paraphrases. A fixed sample of 150 questions not used in the feasibility pilot fro...