Bebe Cosgrove

Hallucination Reduction for LLM-Based Audio Understanding via Multimodal Direct Preference Optimization

Large Audio Language Models (LALMs) are prone to hallucinating and over-relying on text priors when simultaneously presented with audio and text inputs. To mitigate these hallucinations, we propose utilizing the multimodal Direct Preference Optimization (mDPO) objective, which forces the model to ground its generation...