Kevin Miller

Do Audio Language Models Use Paralinguistic Evidence? Counterfactual Audits for Response Evaluation

Audio-language models (ALMs) are increasingly used as judges for speech-to-speech systems, but a judge that receives audio may not actually use paralinguistic evidence. We introduce counterfactual audits for paralinguistic response evaluation. Each audit item holds the transcript fixed while varying affect, prosody, or...