Explainable Sensitive Content Moderation With Vocabulary-Aware Distillation

Abstract

Automated content moderation typically assigns binary safe/unsafe labels without revealing what sensitive behavior appears, who participates, or where in the image it occurs, limiting both auditability and cross-platform adaptation. Grounded explanations require grounded annotations, which existing moderation datasets do not provide at scale. We introduce the Sensitive Benchmark (SenBen), a large-scale scene graph dataset of 13,999 movie frames from 157 films. Each frame is annotated with Visual Genome-style scene graphs (25 object classes, 28 attributes covering affective states such as pain, fear, aggression, and distress, 14 predicates) together with 16 sensitivity tags spanning 5 categories. To evaluate predictions, we define SenBen-Score, a recall-focused composite metric over tags, objects, attributes, and predicates, macro-averaged across categories. All reported scores are measured against single-reviewer-corrected reference labels and are best read as relative comparisons across models on this shared ground truth. A coverage analysis against WordNet shows that 75% of SenBen’s sensitive vocabulary lacks adequate synonym coverage in general lexical resources, motivating a hand-curated domain synonym map released with the dataset. To train a lightweight student model, we propose a multi-task distillation recipe targeting vocabulary imbalance in autoregressive scene graph generation. The recipe combines suffix-based object identity, Vocabulary-Aware Recall (VAR) Loss, and a decoupled Query2Label tag head with asymmetric loss, gaining +5.2 percentage points (averaged over six seeds) in SenBen Recall over cross-entropy training on a 241M-parameter Florence-2-base student. Scaling to Florence-2-large (794M parameters) further raises performance to <inline-formula> <tex-math notation="LaTeX">$\mathrm {R}_{\mathrm {SB}}=0.441$ </tex-math></inline-formula> and <inline-formula> <tex-math notation="LaTeX">$\text {F1}_{\mathrm {SB}}=0.458$ </tex-math></inline-formula>, with precision gains exceeding recall gains as model capacity grows. On grounded scene graph metrics, the base student surpasses every evaluated VLM except Gemini and every commercial safety API, while recording the highest object detection and captioning scores across all models at <inline-formula> <tex-math notation="LaTeX">$7.6\times $ </tex-math></inline-formula> faster inference and <inline-formula> <tex-math notation="LaTeX">$16\times $ </tex-math></inline-formula> lower GPU memory, though relational (predicate) recall remains the principal open challenge.