Performance and Computational Cost of Full and Parameter-Efficient Fine Tuning for Arabic Sentiment Classification Across Training Set Sizes

Abstract

Pre-trained language models are typically adapted to downstream tasks via full fine tuning. However, this entails substantial computational and memory overhead. Parameter-efficient alternatives such as Low-Rank Adaptation (LoRA), Quantized Low-Rank Adaptation (QLoRA), and frozen backbone mitigate these costs, but empirical evidence on how their performance–cost trade-offs change under low labeled data in Arabic remains limited. This paper compares four adaptation strategies: full fine tuning, frozen backbone, LoRA, and QLoRA for Arabic binary sentiment classification on the Hotel Arabic Reviews Dataset, using CAMeLBERT-Mix as the pre-trained encoder. The methods are evaluated under a unified experimental setting at three labeled-data levels: the full training set, 100 samples per class, and 25 samples per class. The evaluation metrics are reported as means and standard deviations across five random seeds. At the full-data level, full fine tuning, LoRA, and QLoRA achieve macro-F1 scores between 0.9569 and 0.9579 and are comparable within seed variability, while the frozen backbone exhibits performance that is approximately ten points lower. LoRA and QLoRA use approximately 35.0% less peak GPU memory than full fine tuning but require longer training times. Under reduced-data conditions, full fine tuning outperforms all other adaptation strategies with the differences being statically significant.