SI-DLG: Scale-Invariant Gradient Matching for Robust Privacy Leakage in Federated Learning

Abstract

Gradient leakage attacks have shown that an honest-but-curious server in federated learning (FL) can reconstruct private training samples directly from the gradients transmitted by clients. The original Deep Leakage from Gradients (DLG) attack, however, suffers from several optimization pathologies: the layer-wise magnitude imbalance of gradients makes the L2 matching loss dominated by a few deep layers, the lack of explicit direction supervision allows gradient orientation to drift during reconstruction, and pixel-space optimization without box constraints introduces high-frequency artifacts. Moreover, the attack largely fails on modern architectures equipped with batch normalization (BN). We propose SI-DLG, a unified gradient inversion framework that addresses these issues with four components: (i) a scale-invariant residual matching loss with a per-layer closed-form optimal scale that analytically removes magnitude mismatch, (ii) a layer-wise cosine direction term that explicitly constrains gradient orientation, (iii) a BN-statistics prior that aligns dummy activations with the running mean and variance cached in BN layers, and (iv) a sigmoid reparameterization that turns the box constraint into an unconstrained problem suitable for L-BFGS with strong-Wolfe line search. Experiments on MNIST, CIFAR-10, and CIFAR-100 with LeNet5-BN demonstrate that SI-DLG achieves high-fidelity reconstruction in terms of PSNR, SSIM, and LPIPS at single-sample reconstruction, and remains effective in the harder batch reconstruction regime (B = 2, 4).