Reliability of subjective performance evaluation in table tennis: Effects of athlete skill level and gender matching

Abstract

This study employed a temporal occlusion paradigm as an experimental control to examine the reliability of observers’ subjective performance evaluations of table tennis players when outcome-revealing visual information was occluded, and to investigate the effects of skill-level and gender matching on evaluation consistency. Ten coaches observed occluded match video clips of table tennis players and provided subjective evaluations of technical-tactical performance across three indicators: stroke quality, tactical behavior, and competitive posture. The observers ( n  = 10) demonstrated high test–retest reliability in their overall evaluations (ICC = 0.76–0.92). Elite-level observers showed consistency in their overall evaluations across athlete skill levels: World Class ( W  = 0.76), Elite Class ( W  = 0.81), and Highly Trained ( W  = 0.73), with the highest consistency observed for Elite Class athletes. At the indicator level, Tactical behavior showed lower consistency, with non-significant agreement for Highly Trained athletes under Score Points ( W  = 0.23, P  = 0.06), and for Highly Trained ( W  = 0.25, P  = 0.67) and World Class athletes ( W  = 0.05, P  = 0.05) under Lost Points. Evaluation consistency was comparable between female and male observers for both female athletes ( W  = 0.79 vs. 0.78, P  = 0.79) and male athletes ( W  = 0.79 vs. 0.81, P  = 0.52), with no significant between-group differences. Observers with professional expertise in table tennis demonstrated high reliability in subjective performance evaluations. Athlete skill level was a key contextual factor, with evaluation agreement highest under skill-level matching and lower under skill-level mismatch. Gender matching did not significantly influence evaluations of observation indicators.