Characterizing Privacy-Audit Alignment in Behavioral Audit of Machine Unlearning

Abstract

The removal of learned data from Machine Learning models through Machine Unlearning (MU) has been widely studied; however, there is no agreed-upon scheme for auditing MU. Existing work shows that a dishonest model owner can falsify evidence to avoid executing MU, while curious auditors (and adversaries) can infer privacy-sensitive properties of the model and its training data even with limited access. Yet auditing of MU under mutual distrust between the model owner and the auditor remains unexplored. In this paper, we characterize how much a generic audit scheme that relies solely on querying the model for behavioral signals inevitably results in privacy leakage related to the retained set by providing a geometric transfer theorem that establishes a lower bound on retained set membership distinguishability based on the audit accuracy. In addition, we study how the unlearned set, target sample, and query protocol jointly determine the privacy-audit transfer coefficient through local and global model-parameter-space geometry. Our empirical experiments on both convex and non-convex models strongly support these results. Our results call for more careful consideration of the privacy-audit tension under a realistic auditor model and serve as a foundation for greater scrutiny of privacy-preserving audit scheme designs for the MU pipeline.