Review Open access

Federated Fine-Tuning of Large Language Models on Resource-Constrained Clients: Technical Approaches, Resource Costs, and Applicability

Yun-Heng Shen Xiao Liu Yang Yang Hai-Rong Lv

Sep 2026 · Journal of Reliable and Secure Computing · 1 citation · 39 references

TL;DR

This review examines four composable routes---parameter-efficient and quantized adaptation, backpropagation-free adaptation, proxy or submodel adaptation, and split federated adaptation---through a common framework that traces the objects each endpoint retains, exchanges, and discloses.

Abstract

Federated fine-tuning adapts language models to distributed data and is widely adopted as a privacy-preserving alternative to centralized training, yet constrained clients must still store model weights and training states, execute updates, and communicate with a server. This review examines four composable routes---parameter-efficient and quantized adaptation, backpropagation-free adaptation, proxy or submodel adaptation, and split federated adaptation---through a common framework that traces the objects each endpoint retains, exchanges, and discloses. When a client retains the complete base, reducing adapter state leaves a base-storage floor; boundary communication depends on input dimensions, sequence length, and interaction frequency, so parameter count alone is insufficient. Keeping data local constrains the raw dataset but not the transmitted object: adapter updates, boundary activations, logits, and cached embeddings expose different surfaces, and the encryption, secure-aggregation, or noise mechanisms that protect them add to the same budgets. Server weight sharing removes duplication but retains private states and caches, and combining routes requires recalculating these costs for the resulting protocol. The analysis identifies where savings arise, where costs move, what each route discloses, and which training, deployment, and trustworthiness conditions govern its applicability.

DOI Semantic Scholar

Read PDF

Similar papers

Preprint Aug 2026

Thermo-FL: Thermal-Aware Robust Federated Fine-Tuning of Large Language Models for Edge AI

Thermo-FL is presented, a thermal-aware federated LoRA fine-tuning framework that uses device temperature as an active control signal for local adapter training and sparse update transmission and introduces TERRA, a robust aggregation pipeline for dynamically sparse LoRA updates.

S. Shrestha, K. Sharif, Zongxing Xie et al. · 0 citations

#machine learning Preprint Sep 2026

Privacy-Preserving Split Learning for Federated LLM Fine-Tuning

This work addresses leakage through a learned obfuscate-and-recover scheme that protects participants' private datasets while still allowing an independently deployable model to be trained on the server side, making split-based federated LLM fine-tuning practically viable.

Heng Jin, Chao-Yu Zhang, He-Xuan Yu et al. · 1 citation

#federated learning Open access Oct 2026

Overcoming Memory Bottlenecks in Homomorphic Federated Learning: A Chunk-Based Serverless MapReduce Approach

Federated Learning (FL) enables collaborative model training across decentralized organizations without direct data sharing, yet communicating raw gradient updates remains susceptible to reconstruction and membership inference attacks. Fully Homomorphic Encryption (FHE) provides cryptographic privacy during aggregation...

Afonso Cruz, Carlos Marques, Vasco Carvalho et al. · Big Data and Cognitive Compu... · 0 citations

#artificial intelligence Preprint Sep 2026

Task-Aware Federated Fine-Tuning for MoE-based Large Language Models

Mixture-of-Experts (MoE) has become a widely adopted architecture for Large Language Models (LLMs), as it improves model capacity while limiting computational overhead through sparse expert activation. This property makes MoE-based LLMs particularly attractive for resource-constrained distributed environments. However,...

Ting-Qi Wang, Hongyu Ke, Hao-Xin Wang et al. · 0 citations

#machine learning Preprint Sep 2026

Backdoor Mitigation in Decentralized LLM Fine-Tuning

Decentralized large language model (LLM) fine-tuning lets organizations collaboratively train a shared LLM on data they cannot pool, without a central coordinator. In every round, each node exchanges a trainable adapter with its neighbors over a communication graph, and then aggregates them. This setting, however, is v...

Sayan Biswas, Jade Garcia Bourrée, R. Guerraoui et al. · 0 citations

#large language models Preprint Sep 2026

Just Talk Once: Communication-Efficient Split Federated LLM Fine-Tuning on Edge Devices

L-shaped SFT is presented, a split fine-tuning framework that removes the need for continuous client participation and introduces one-shot SFT, in which clients upload activations once and then go offline while the server continues optimization over cached representations.

Jia-Xiang Geng, Xian-Hao Chen, Bing Luo · 1 citation