Gradient Mirage: Trainable yet Label-Unidentifiable Gradients in Large Language Model Split Learning
The key idea is to induce the adversary to solve a misspecified inverse problem, in which no plausible label sequence in the sequence space can explain the observed gradients, by inducing inconsistency across three dimensions: objective, direction, and scale.