Multi-Scale Contextual Alignment and Sequence-Adaptive Attention for Video Summarization
TL;DR
MASA, a parameter-compact framework built around Multi-Scale Contextual Alignment and Sequence-Adaptive Attention for frame-level importance prediction, supports competitive rank-correlation performance with an 11.44M-parameter scoring model, while the measured computational cost shows that parameter compactness should not be interpreted as low FLOPs.
Abstract
Video summarization aims to condense long videos into compact and informative representations, but existing sequence-modeling and attention-based methods still face challenges in handling heterogeneous temporal rhythms, long-range semantic drift, and redundant background frames. This paper presents MASA, a parameter-compact framework built around Multi-Scale Contextual Alignment and Sequence-Adaptive Attention for frame-level importance prediction. MASA represents sampled videos as channel-replicated temporal-feature tensors; the replication provides an input-compatible form for the convolutional backbone but introduces no new spatial information. Multi-Scale Contextual Alignment constructs a video-conditioned prototype anchor and aligns local sequence patterns at complementary temporal receptive fields. Sequence-Adaptive Attention predicts a position-dependent Gaussian aperture, and Dual-Axis Feature Aggregation combines temporal and latent-feature responses before score prediction. Under the fixed five-fold protocol, MASA achieves Kendall’s tau/Spearman’s rho of 0.263/0.294 on SumMe and 0.196/0.258 on TVSum. A repeated-run study reports run-level variability and confidence intervals for representative component and kernel configurations, and the full model maintains stable advantages over the strongest same-code ablations on most matched runs. These results support competitive rank-correlation performance with an 11.44M-parameter scoring model, while the measured computational cost shows that parameter compactness should not be interpreted as low FLOPs.