It is reported that the nonuniform semistructured framework detects critical layers and that, as demonstrated through successive pruning experiments, their preservation from pruning greatly reduces performance degradation, compared to a fully pruned model.
This systematic literature review (SLR) provides a comprehensive overview of pruning techniques applied to LLMs, based on 60 peer-reviewed studies and preprints published between 2022 and 2025, sourced from major digital libraries.
F. Bazikar, Atefeh Hemmati, Akram Reza et al.· Knowledge and Information Sy...· 0 citations
It is discovered that outlier dominance grows with model scale: full replacement at K=128 degrades GPT-2 small but catastrophically degrades Mistral-7B by +1,134,279%, while selective replacement at p=99% rescues both models to under +15%.
Mask2Shield (M2S), a masked-forward alignment method that trains a model under this functional pruning procedure, reduces successful recomputed pruning attacks from 80--279 to 1--44 out of 313 prompts while generally preserving four capability benchmarks.
SNIPER is presented, a two-stage structured pruning framework that solves a knapsack optimization over coarse-granularity components to yield conditionally optimal parameter allocations with respect to fixed importance estimates, followed by a fine-grained pruning stage to meet strict budget constraints.
Palaash Goel, Ayan Sengupta, A. Nambi et al.· 0 citations
PALS (Percentile-Aware Layerwise Sparsity), which adjusts per-layer sparsity based on the 99th percentile of activation magnitudes, bounded to $\pm 5\%$ around the target ratio, is proposed.
Yazdan Jamshidi, Alexey A. Shvets· arXiv.org· 0 citations
