This paper studies what happens to the rest of the model when a class is forgotten, using a label-conditioned energy-based model (EBM) that assigns per-class energies, making the effect directly observable.
Abstract
Machine unlearning removes the influence of specific data from a trained model. However, most methods treat the forgotten concept as isolated. In this paper, we study what happens to the rest of the model when a class is forgotten, using a label-conditioned energy-based model (EBM) that assigns per-class energies, making the effect directly observable. We forget a class by raising the energy of its image-label pairs, training with a forget term, a retain anchor to the pretrained model, a global margin, and an energy regularizer that stops the energy magnitudes from growing without limit. A propagation term applies the same forget signal to retain samples, weighted by each sample's DINOv2 similarity to the forget class, so forgetting reaches images that resemble it and leaves the rest untouched. We evaluate on two benchmark datasets: 1) On a subset of DomainNet across four visual domains, we forget tiger, lion, and scissors one at a time. Forgetting a class in the sketch domain also erases it from real, clipart, and painting, with forgetting error reaching 98% and 99% for lion and scissors, and the effect carrying over to the most similar class. 2) On CIFAR-10, we turn off the propagation term and forget each of the ten classes on its own. Forgetting is complete (100%), while the other nine classes retain 98.5% of their pre-unlearning accuracy on average.
Machine unlearning (MU) aims to remove the influence of selected data from trained models, offering an efficient alternative to full retraining. With the rise of increasingly stringent privacy regulations, including the right to be forgotten, machine learning models must incorporate mechanisms that ensure compliance while preserving performance. However, existing MU methods often suffer from performance degradation, and limited granularity, leaving residual traces that adversaries can exploit. We propose DeepU, a fine-grained unlearning framework that performs within-layer, weight-level analysis and quantifies the influence of individual weights using the signal-to-noise ratio (SNR) derived from the gradients of sensitive and non-sensitive data. DeepU categorizes weights as influential, intra-dependent, or non-influential and applies targeted weight updates: resetting or perturbing influential weights, decaying intra-dependent ones, and stabilizing the remaining ones via limited backpropagation. We evaluated DeepU on standard datasets including CIFAR-10, CIFAR-100, Tiny ImageNet, and CelebA, demonstrating a 60–90% reduction in successful membership inference attacks with less than a 3% drop in accuracy, significantly outperforming existing benchmark methods. Re-tuning requires only 20.75 seconds and 102.47 MB, making DeepU up to 36.6 times faster than competing methods. We further evaluate robustness against Generative Adversarial Network (GAN) and diffusion-based model inversion attacks across white-box, black-box, and label-only threat models, demonstrating that DeepU consistently reduces reconstruction of forgotten data.
Machine unlearning aims to remove the influence of specific training data while preserving model utility. Many state-of-the-art approaches pursue this goal by restricting the forgetting update to a subset of parameters selected through gradient-based saliency. Although such methods are widely adopted, the actual contribution of saliency-based weight selection to representation-level forgetting remains unclear. In this work, we perform the first controlled ablation of the saliency masking mechanism used by SalUn. Using a matched-compute experimental design on CIFAR-10 and CIFAR-100 with ResNet-18, we compare saliency-based masking against random masks of equal sparsity and unconstrained updates, while keeping the unlearning objective, optimization schedule, and computational budget fixed. Across multiple representation-level evaluations, including linear probing, prototype recovery, and layer-wise CKA, the three configurations exhibit statistically equivalent representation-level recoverability. We find that forget gradients are strongly concentrated in the final network layers (approximately 92% of the squared gradient energy on CIFAR-10) before any mask is applied, causing all masking strategies to operate within the same representational subspace. Furthermore, saliency masks show limited class specificity (specificity index 0.09-0.11), selecting highly overlapping parameter subsets across different forget classes. Our findings suggest that, in the studied setting, representation-level forgetting is primarily governed by gradient concentration and representation geometry rather than by the specific identity of saliency-selected weights. More broadly, the results support a growing body of evidence indicating that effective representation-level unlearning requires objectives that act directly on latent representations rather than on increasingly sophisticated weight-selection strategies.
Billel Habbati, A. Merlo, Luca Verderame et al.· 0 citations
Machine unlearning has become a critical capability for safely removing specific, sensitive knowledge from large language models (LLMs). Current state-of-the-art approaches primarily rely on iterative, training-time unlearning via fine-tuning. However, even when utilizing parameter-efficient dimensionality reduction techniques like LoRA, gradient-based optimization remains computationally expensive and lacks explicit analytical formulations. It can also leave the targeted knowledge merely hidden rather than removed, to the point that simply quantizing the unlearned model restores much of what it was supposed to have erased. To resolve this, we propose a novel one-shot unlearning approach, abandoning iterative optimization in favor of a direct, exact analytical solution. We frame the unlearning process as a ridge-regularized least-squares optimization problem, deriving a closed-form additive update for targeted weight matrices. This update forces the selected layer to suppress unwanted content while strictly preserving its behavior on retained data. Computed from gradient-free forward passes alone, with no backpropagation and no iteration to convergence, GROM applies the weight edit in mere seconds, which makes it orders of magnitude faster than traditional fine-tuning. Extensive evaluations demonstrate that GROM achieves state-of-the-art forgetting-utility trade-offs on TOFU-5%, TOFU-10%, MUSE-Books, MUSE-News and WMDP, significantly reducing computational overhead without sacrificing overall model performance. Because the update removes the targeted content from the weights instead of masking it, GROM also withstands the low-bit quantization attack that recovers much of the content a gradient-based baseline had appeared to forget. Our code is publicly available at https://github.com/Batorskq/GROM.
Paweł Batorski, P. Spurek, Paul Swoboda· 0 citations
Class-wise Covariance Regularization is proposed, which aligns the predicted covariance structure of class confidences with the semantic correlations encoded in pretrained text embed-dings with the geometric consistency of the class space throughout fine-tuning, resulting in more stable and interpretable confidence distributions across categories.
Ao Zhou, Zhiwei Jiang, Zifeng Cheng et al.· 0 citations
With the continual advancement of computer vision techniques, the efficacy of deep learning models has markedly improved. Nonetheless, these accomplishments depend on the premise that both training and test data adhere to the independent and identically distributed (i.i.d.) property. This assumption is frequently violated in practice due to the variability of unobserved data distributions, resulting in performance deterioration. These variations are named domains, and this has motivated the study of robustness to domain shifts, commonly referred to as domain generalization (DG). In this context, DG methodologies have emerged as a novel research paradigm. This research introduces a domain generalization framework, MISA (Mutual Information-driven Separator with Spectral Alignment), which disentangles and learns semantic features via mutual information optimization and spectral alignment. MISA incorporates an attention mechanism and a domain classifier to extract domain information, while three complementary loss functions directly promote feature disentanglement. Additionally, we conceptually establish that the proposed components reduce generalization error in unseen domains and empirically verify the efficacy of MISA by comprehensive comparisons with existing methodologies.