Are You Sure You're Sure? Two Confounds in a Sycophancy Benchmark
An audit of SycEval, which reports that objections raised before a model answers (preemptive) cause more caving than those raised after (in-context), and attributes the gap to timing, and builds the missing comparison and tests it on multiple-choice questions and SycEval's own free-form pipeline.
Atharva Gupta, A. Jindal, Lavanya Nigam et al.
· 1 citation