Kwok-Wai Cheung

StructTune: Structured Sparse Fine-Tuning for Efficient Adaptation of Large Language Models

This work introduces StructTune1, a PEFT method that uses entire rows or columns as basic units for identifying task-relevant sparse components and develops a sensitivity-based framework to efficiently identify task-relevant weight structures without full fine-tuning.