Open-set Visual Template Discovery in Large-Scale Propaganda Image Collections

Abstract

Reusable visual templates are common in large image collections, where layouts, color schemes, typography, and design elements are repeatedly adapted while only local content changes. Prior work typically treats template analysis as a closed-set clustering problem in which the number of categories is known and evaluation is performed on manually selected examples. We reformulate the task as open-set template discovery, where coherent template families must be recovered from a mixed corpus without knowing K while treating non-template content as noise. We evaluate frozen encoders on a 398-image labeled benchmark derived from Taiwan’s 2021 referendum corpus and on the full 9,354-image collection. On the labeled benchmark, DINOv2 reaches ARI = 0.894 with K-means clustering and 0.879 with automatic-K Ward clustering, approaching an in-domain-trained pixel autoencoder (ARI = 0.910) without in-domain training. On the full corpus, a CLIP-DINOv2 HDBSCAN ensemble retains 1,878 images, covers 80.9% of labeled instances, and correctly assigns 80.4% of instances to plurality-consistent clusters. The two encoders exhibit complementary behavior: DINOv2 more reliably recovers visual-style families, whereas CLIP better preserves screenshot-like and text-layout templates. Based on the observed recovery patterns, we propose an empirical taxonomy of visual, style, and semantic templates that identifies which categories are recoverable through visual clustering and which require higher-level semantic signals, providing empirical insight into the limits of purely visual template discovery.