Skip to content
Book Open access

TextFilter: Filtering Semantic-Temporal Representations with Texts for Video Action Recognition

Nov 2026 · Proceedings of the 34th ACM International Conference on Multimedia · 0 citations · 7 references

Abstract

Vision–Language Models (VLMs) have advanced video action recognition, yet most existing approaches still rely on text as a static semantic cue. In real video data, however, the primary challenge often lies in the noise (dominant backgrounds, scene biases, and temporally irrelevant motions) that obscure discriminative patterns. We build on the mixture-of-experts framework and propose TextFilter, which repositions language from a passive descriptive signal to an active mechanism for semantic-temporal filtering. Rather than solely injecting semantics, TextFilter uses text to suppress misleading visual signals. A text-guided background attenuation module first reduces semantic interference by suppressing background signals. In parallel, text-driven semantic and temporal Mixture of Cross-modal Experts (MoCE) modules leverage fine-grained LLM-generated descriptions to guide expert selection. The semantic branch focuses on object-relevant regions, while the temporal branch emphasizes action-critical motion phases. Together, these components yield cleaner representations, more reliable routing, and stronger video–text alignment. We assess TextFilter across five widely used benchmarks: Kinetics-400/600, UCF-101, HMDB-51, and SSv2. Our TextFilter delivers competitive performance compared to state-of-the-art methods in zero-shot, few-shot, and fully supervised regimes, while remaining highly parameter-efficient.

Read PDF

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.