Six Methodological Pitfalls in YouTube Comment Extraction: What an interface constraint does to a research corpus

Abstract

If your study uses YouTube reply structure — who answers whom, how deep a thread runs, which turn follows which — this paper shows where that structure is not what it seems, and gives a check for each problem that runs in minutes on any comment corpus. Nine diagnostics, a documented mention-resolution rule and the scripts that produced every figure are included. The central finding is a hard nesting cap. Interface-level extraction returns four levels of comment depth in total, depths 0 to 3, and no more: across 210 extractions of 133 videos (480,925 comments, 238,700 replies), not one comment reached depth 4, and in 55 extractions the deepest level is the largest. The consequence is sharp. At depth 2, the address marker the platform inserts and the recorded parent agree in 13,404 of 13,407 cases; at depth 3 they disagree in 45.8%. Edges at the deepest level are forced attachments, not replies to the comment named, and anything computed on them without a depth control measures the platform rather than the speakers. Further failures are measured, each with its effect size: timestamps rebuilt from coarse labels (143,421 comments on 37 distinct time values); handle migration, which leaves 7.9% of address markers unresolvable and fakes part of a trend if they are scored as divergence; orphaned replies re-attached after deletion; collectors that silently drop every reply while the parent field stays populated (258 videos, 89,074 comments in the author's own collections); and invisible characters that hide 3–4% of address markers from a standard pattern match. The sixth pitfall is methodological. A manual coding pass, specified in advance and correctly executed, confirmed 78 of 100 divergent cases as genuine, and the conclusion drawn from it was wrong: case-level coding cannot see a mechanism that operates at the level of tree position. The remedy costs one grouped count.