Counterfactual Tests for Measuring Chain-of-Thought Faithfulness in Visual Language Models
The analysis shows that CoTs do not reliably track visual evidence that influences model predictions, and it is found that Predict-then-Explain explanations align more strongly with perturbation-induced probability shifts than pre-answer CoTs, while binary vCT scores are often nearly saturated.
Bayar Menzat, Max Süss, Rui-Zhi Wang et al.
· 0 citations