Toward Multimodal AI for Dementia: A Challenge-Driven Survey of LLMs and VLMs

Abstract

Dementia affects over 57 million people worldwide and places an immense burden on informal caregivers, yet current AI tools remain largely fragmented across isolated tasks and modalities. Recent large language models (LLMs) and vision-language models (VLMs) offer promising capabilities for dementia support, but adapting them to this safety-critical, multimodal, and deeply individualized care domain raises challenges that general-purpose AI surveys do not address. In this paper, we present a challenge-driven survey that organizes the rapidly growing literature on LLMs and VLMs for dementia care around three core adaptation challenges: (1) Knowledge Grounding, which anchors model outputs to verified clinical knowledge through retrieval augmented generation, knowledge graphs, and constrained training to mitigate hallucination risk; (2) Multimodal Understanding, which fuses visual, audio, and sensor data with language to reason about the multimodal inherent of daily care; and (3) Personalization, which adapts model behavior to individual patient histories, caregiver needs, and unique disease progression over time via persistent memory and biography-driven interaction. We review over 20 recent methods, identify cross-cutting architectural patterns, and survey available datasets and benchmarks. Finally, we highlight critical open challenges including the need for standardized evaluation protocols, longitudinal deployment studies, and tighter integration between clinical workflows and foundation model capabilities.