CONDITION NUMBER-AWARE PRUNING: PRESERVING MATHEMATICAL STABILITY IN SPARSE NEURAL NETWORKS

Abstract

The increasing scale of deep neural networks has necessitated model compression techniques, with pruning emerging as a prominent approach to reduce computational and memory costs. However, aggressive pruning introduces a critical challenge: the degradation of mathematical stability and adversarial robustness. Recent research reveals that highly pruned weight matrices tend to become ill-conditioned, exhibiting exploding condition numbers that undermine model performance and robustness . This paper proposes a condition number-aware pruning framework that explicitly preserves mathematical stability during the pruning process. We establish theoretical connections between sparsity, condition number, and local Lipschitz continuity, demonstrating that the condition number becomes the dominant factor limiting robustness in over-sparsified models . Our methodology integrates a differentiable Condition Number Constraint (CNC) with transformed sparse regularization (TSCNC) to simultaneously achieve high sparsity and well-conditioned weight matrices. Experimental evaluations on CIFAR-10, CIFAR-100, and Tiny-ImageNet demonstrate that our approach significantly improves both standard accuracy and adversarial robustness compared to conventional pruning methods, achieving superior performance across VGG, ResNet, and WideResNet architectures.