Adnane Bakkou

Structured Pruning of Small Language Models: An Empirical Study on GQA-Aware Attention and MLP Compression with LoRA Recovery

Large language models have shown strong performance across many natural language tasks, but their size makes deployment difficult on hardware with limited memory and compute. Compression techniques aim to reduce this cost while preserving as much quality as possible. This study investigates structured pruning applied t...