Replicability: Challenging the Myth of Randomness Patterns in Encrypted Traffic

Abstract

Several recent studies claim that encrypted network traffic exhibits statistically significant randomness deficiencies due to differences in cryptographic algorithms, implementations, and parameterization, and that such patterns can be exploited for application classification in TLS traffic. This interpretation has been adopted by measurement researchers, most prominently in the often-cited ET-BERT study. We present a systematic replication of these randomness experiments, re-implementing them using the NIST Statistical Test Suite on encrypted traffic generated under controlled conditions. In the replicated work, the suite is not itself a detection component, but is used to argue that detectable patterns exist in ciphertext and thereby motivate the method. It is this evidentiary step that we examine. Our evaluation shows that the reported statistical signals do not provide sufficient evidence of non-randomness beyond what is expected from statistical hypothesis testing and finite-sample effects. The observed test failures and discriminative patterns are consistent with methodological artifacts, including improper aggregation, p-value misinterpretation, and information leakage between training and test data. Overall, our findings suggest that assumptions about imperfect randomness in encrypted network traffic are unsubstantiated. Our critique is limited to randomness-based classification, and we make no claim about methods exploiting metadata, packet sizes, timing, or other side-channel information.