Detecting Pretraining Data in Large Language Models from a Free-Energy Perspective
This work introduces an inclined boundary that evaluates prediction loss relative to predictive entropy, and shows that entropy correction can preserve the expected membership signal while reducing its variance, thereby improving standardized member--non-member separation.
Chen-Ye Ke, Zi-Rui Liu, Qi Liu et al.
· 0 citations