NewsSegCap: Efficient Dense Video Captioning for TV News

Abstract

Automatic dense video captioning remains a challenging task, particularly for long and information-rich content such as TV news. However, the highly structured format of TV news programs offers opportunities for efficient segmentation strategies. In this work, we show how diarization-based segmentation can exploit this structure to reduce reliance on computationally heavy visual processing. The proposed NewsSegCap approach includes two specialized modules. The first one, called NewsSeg, leverages solely presenter diarization and a fine-tuned Qwen2.5-7B-Instruct model to segment the video into segments that correspond to a unique, unitary thematic topic. By analyzing only the presenter sentences, it significantly reduces the number of speech tokens to process, from approximately 10,000 to 2,600 for a typical 38-minute program. The second module, called NewsCap, is a segment-level captioning approach based on a Qwen2.5-VL-7B model fine-tuned using only LoRA. It integrates multimodal inputs, including visual features, speech transcripts, and OCR-derived cues, to generate context-aware captions. The proposed approach is validated on a refined version of the MediaEval2015 dataset, which consists of 140 TV news videos with human dense annotations. To address data scarcity, we also introduce TV5News, a weakly supervised segmentation pre-training dataset, containing 489 French news broadcasts diarization, with segmentations distilled from Gemini. The proposed approach achieves strong segmentation performances, with mF1 and tIoU scores of 83.2 and 87.8 % respectively, which are competitive to those obtained by very large proprietary models. The NewsCap module produces dense, semantically rich captions that capture both thematic and visual aspects of TV news segments, outperforming Gemini-3-Flash model.