Stabilized Fine-Tuning with LoRA in Federated Learning: Mitigating the Side Effect of Client Size and Rank via the Scaling Factor

Stabilized Fine-Tuning with LoRA in Federated Learning: Mitigating the Side Effect of Client Size and Rank via the Scaling Factor explores SFed-LoRA stabilizes LoRA fine-tuning in federated learning by mitigating gradient collapse, enabling faster convergence and improved stability, making it a practical solution for privacy-preserving LLM adaptation.. Commercial viability score: 7/10 in Federated Learning.