TL;DR
This work proposes a compressed-video-oriented framework, the Frame Selection and Token Merging for Efficient Compressed Video Transformer (FSTM-ECVT), which follows a dual-stream Transformer architecture equipped with a Global Multi-Modal Fusion module to effectively leverage the distinctive and complementary characteristics of RGB and compressed motion modalities.
ENCORE, an Event-Assisted Complementary Motion Refinement framework for learned video compression, employs Complementary Motion Representation to decompose aligned RGB-event features into common and modality-specific motion representations and identifies event-specific responses that are active and novel relative to RG...
Video saliency prediction aims to estimate the regions in dynamic scenes that are most likely to attract human attention. Although Transformer-based backbones have demonstrated strong performance in spatiotemporal representation learning, existing decoders still rely primarily on convolutional operations, which are lim...
KATok (Keep-or-Drop? Adaptive Tokenizer for Compact Video Representation), a transformer-based VAE that incorporates an adaptive token selector which is jointly learned with latent tokens that achieves strong reconstruction and generation quality at a state-of-the-art compression ratio.
Event cameras can capture human motion with low latency and high dynamic range, providing an emerging sensing paradigm for action recognition. A large body of recent work converts event data into a sequence of dense frames and feeds them into a video classification model for prediction. Although the frame-based methods...
We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.
