LingVox: A Multimodal Conversational Corpus Analysis Platform

Abstract

Conversational and multimodal corpora are central to affective computing, yet their analysis remains challenging due to heterogeneous data formats, modality-specific signals, and interactional dynamics such as speaker turns and temporal variation. Existing approaches often rely on ad hoc toolchains, making workflows difficult to reproduce, compare, and scale. This is becoming a challenge given the rapid advancements in AI models capable of richer affective analysis. We present LingVox, a web-based platform for structured, reproducible multimodal conversational corpus analysis. LingVox decomposes inputs into individual analysis units: speaker-turn segments for text, timestamped segments for speech audio, and keyframes for video. It applies modality-specific affect analysis, including text-based sentiment and escalation modeling, acoustic-aligned affect scoring, and zero-shot CLIP-based visual affect estimation. Using locally deployed LLMs via Ollama, LingVox generates interpretable summaries grounded in metrics while reporting runtime and estimated CO2 emissions via CodeCarbon. The platform unifies disparate pipelines into a plug-and-play workflow, bridging raw multimodal data and high-level affective insights. LingVox’s interface allows for visual and comparative inspection of model-driven analysis and study of human interaction, while also providing energy efficiency measurement insights.