TraVIS: Combining Player Tracking and Video for Football Action Spotting

Abstract

Action spotting aims to localize actions in long, untrimmed videos using a single keyframe per action, making it well suited to the fast-paced nature of sports. Focusing on football, we identify two remaining challenges. First, although existing methods account for different temporal context lengths and scales, their relative effectiveness has not been systematically compared. Second, most existing methods rely exclusively on video features, requiring models to infer football-specific spatial and relational structure implicitly from appearance and motion, which is particularly challenging when only a limited number of annotated matches are available. We address these limitations through two contributions. First, we conduct a comparative analysis of single- and multi-scale temporal architectures, showing that multi-scale models provide only modest gains over single-scale alternatives in video-only settings relative to their additional computational cost. Second, we introduce TraVIS, an action spotting framework that complements video features with explicit player and ball tracking information. TraVIS represents the game state as a graph whose nodes encode players and the ball and whose edges capture their spatial and relational structure, thereby introducing football-specific inductive biases and explicitly representing information that video-only approaches must otherwise learn implicitly. Fusing the graph and video representations achieves the strongest performance among the evaluated configurations on a proprietary football dataset, improving over the video-only model by + 6.42 mAP@1s. These results show that structured tracking information provides a valuable complement to video features for football action spotting.