Jul 2026· International Conference on Ubiquitous and Future Networks· pp. 1230-1233· 0 citations· 6 references
Abstract
Knowledge distillation (KD) enables efficient deployment of deep learning models. However, it can increase privacy risks by making the student more vulnerable to membership inference attacks (MIAs), as the student may inherit patterns specific to the teacher’s private training data. Existing KD-based methods against MIAs often rely heavily on reference data or require multiple teachers, which limits their practical applicability. To address this issue, we propose Teacher-Anchored Worstcase Stability Distillation (TASD), a practical single-teacher KD method for reducing membership inference risk. Experiments on CIFAR-10 show that TASD provides stronger resistance to MIAs than existing KD-based methods at lower computational cost.
Knowledge distillation (KD) has become a pivotal technique for transferring knowledge from large-scale teacher models to lightweight student models. However, traditional feature-based distillation methods necessitate the direct exposure of the teacher’s intermediate representations, raising concerns regarding data priv...
Jun-Fei Yi, Si-Hao Lin, Hui Zhang et al.· IEEE Transactions on Image P...· 0 citations
FISGuard reduces the ProjRes attack AUC to near the random-guessing level of 0.5 in most settings, while maintaining downstream task performance close to that of the undefended model and introducing only limited computational overhead, thereby achieving a favorable privacy--utility trade-off.
P2Skill is proposed, a prompt-based skill distillation method in which a local small language model (SLM) autonomously performs decomposition, PII-aware routing, paraphrasing, and reconstruction by following the skill prompts.
M. Ryu, Geunpyo Park, Sungjoon Lee et al.· 1 citation· ⚡1
SQuaT (Student-Aware Quantized Teacher Features), a label-free QAT framework with KD that theoretically eliminates this lower bound on the distillation loss by applying the student's quantization parameters to quantize the teacher's features during distillation is proposed.
H. Lee, Hyeonsik Jo, Jinwook Chung et al.· 0 citations
This work introduces an attention-driven dynamic differential privacy mechanism that enables adaptive allocation of privacy budgets and builds a lightweight multi-head attention component that dynamically evaluates each feature’s relevance and the current phase of training, adjusting the intensity of injected noise on...
Hai-Ning Shang· Frontiers in Computing and I...· 0 citations
Federated Learning (FL) avoids centralizing raw data, but server-side access to per-client updates still creates a significant privacy risk because gradients can leak sensitive information through inversion and related attacks. A common defense is client-level Differential Privacy (DP), which reduces attack fidelity by...
Clifford N. Jones, Md Nahid Hasan, S. Wagle et al.· International Conference on...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
