Unveiling the Depth-Performance Dilemma in Split-Federated Fine-tuning of LLMs
This work identifies and characterize the Depth-Performance Dilemma: the regime that maximizes system efficiency is precisely where fine-tuning quality collapses, and challenges the prevailing assumption that partition depth is a utility-neutral tuning knob and provide a structural foundation for stable distributed LLM...