Adaptive BitFit: Gradient-Driven Sparse Bias Fine-Tuning for Efficient Cancer Classification

Abstract

Accurate histopathological classification is essential for early cancer diagnosis, yet deep learning models often require extensive computational resources, limiting their deployment in real-world clinical settings. Parameter-Efficient Fine-Tuning (PEFT) strategies offer a promising alternative, but existing methods such as LoRA and adapter-based approaches still update a substantial number of parameters. This study introduces Adaptive BitFit, a sparsity-driven fine-tuning framework that selectively updates only the most informative bias parameters using gradient-based importance profiling. Integrated with MobileViT and Vision Transformer backbones, the proposed method significantly reduces computational overhead while preserving high diagnostic accuracy. On the LC25000 dataset, standard BitFit alone achieved up to 98.18% accuracy with a comparable F1-score, demonstrating the potential of bias-only tuning even before applying the proposed framework. Extensive experiments on LC25000, NCT-CRC-HE-100K, and the CRC-VAL-HE-7K held-out validation set (no patient overlap with the training NCT-CRC-HE-100K but from the same source study) demonstrate that Adaptive BitFit achieves up to 60% reduction in trainable bias parameters with minimal accuracy loss, attaining 97.86% accuracy on LC25000 and 96.99% on CRC-VAL-HE-7K. Also the model with 20% sparsity level, achieved around 92% accuracy and F1 score on a brain cancer MRI dataset, which proves the generalizability of this study. Comparative analyses against Adapter, BitFit, LoCon, and SSF confirm the superior balance between efficiency and performance. Furthermore, Grad-CAM visualizations enhance interpretability by highlighting clinically relevant tissue structures. These results position Adaptive BitFit as an effective, lightweight, and generalizable solution for deploying deep learning models in resource-constrained medical environments. Code Availability: https://github.com/N-Kibria/PEFT-CancerX