PALS: Percentile-Aware Layerwise Sparsity for LLM Pruning

Yazdan Jamshidi Alexey A. Shvets

Jul 2026 · arXiv.org · Vol abs/2607.07557 · 0 citations · 17 references

Computer Science

TL;DR

PALS (Percentile-Aware Layerwise Sparsity), which adjusts per-layer sparsity based on the 99th percentile of activation magnitudes, bounded to $\pm 5\%$ around the target ratio, is proposed.

Abstract

One-shot pruning methods like Wanda and SparseGPT apply the same sparsity ratio to every layer of a transformer, ignoring known variation in layer importance. We propose PALS (Percentile-Aware Layerwise Sparsity), which adjusts per-layer sparsity based on the 99th percentile of activation magnitudes, bounded to $\pm 5\%$ around the target ratio. On LLaMA-2-7B at 50\% sparsity, PALS achieves 10.96 WikiText-2 perplexity versus 12.92 for uniform Wanda (mean over 9 runs, $p<0.001$). The benefit is architecture-dependent: LLaMA-3-8B shows marginal gains and Mistral-7B shows none. We also find that gradient-based allocation -- the seemingly more principled approach -- produces results worse than random, suggesting that gradient magnitude does not predict the impact of discrete weight removal. PALS adds negligible cost to the pruning pipeline and requires no fine-tuning.

DOI arXiv Semantic Scholar

View source

Similar papers

Sep 2026

Fisher-Based Layer-Wise Adaptive Sparsification for Efficient Pruning of Large Language Models

Chuan Sun, Han Yu, Xiao-Xiao Li · IEEE Transactions on Mobile... · 0 citations

#natural language process... Preprint Sep 2026

Debias-SparseGPT: Bias-Aware Pruning for Large Language Models

Debias-SparseGPT is introduced, a post-training pruning method incorporating representational debiasing using a second-order term defined over demographically contrasting inputs that consistently reduces pruning-induced bias compared to SparseGPT while preserving model perplexity and zero-shot accuracy.

Irina Proskurina, Guillaume Metzler, Antoine Gourru et al. · 0 citations

#artificial intelligence Preprint Aug 2026

QTEA: Ternary LLMs with Sparse Residual Salient Weight and By-Column Optimization

This work presents QTEA, a sub-2-bit PTQ framework that quantizes weights into ternary values and uses salient weights as residual error compensators and develops a lookup-table based kernel that achieves 7.2% faster per-token generation over an FP16 baseline.

Yi-Pin Guo, Arun George, Jie Fu et al. · 0 citations

#machine learning Preprint Sep 2026

MoARa: Module-Aware Rank Allocation and Structure-Preserving Decomposition for Low-Rank LLM Pre-training

Low-rank gradient projection reduces the optimizer-state memory cost of large language model (LLM) pretraining, but the steps and wall-clock time needed to reach a target quality remain a meaningful axis for improvement. We attribute this to two design choices in existing methods: the projection-rank budget is allocate...

Keun-sik Kim, N. Kwak · 0 citations

Preprint Aug 2026

When Pruning Meets Interpretability: Preserving Sparse Autoencoder Robustness in LLMs

A systematic study of how pruning affects SAE behavior is presented and theoretically shows that, for a fixed SAE, its impact is governed by perturbation energy, a covariance-weighted norm.

Suchit Gupte, Xue-Ru Zhang, M. Khalili · 0 citations

Opleiding DSAI Pruning-aware Shapley values estimation for Large Language Models

It is reported that the nonuniform semistructured framework detects critical layers and that, as demonstrated through successive pruning experiments, their preservation from pruning greatly reduces performance degradation, compared to a fully pruned model.

Ann Marini, Qinyu Chen, Nusa Zidaric · 0 citations